Natural Language Processing and Recommendation Engine for Stack Overflow Data

Abstract

Query intent classification is important for information retrieval and problem solving. We use natural language processing and collaborative filtering algorithms to build a recommendation engine for Stack Overflow tag predictions. Our pipeline consists of document retrieval (TF-IDF and HOTT), text embedding (Sentence BERT), and classification (multi-label and multi-class). We experiment with neural networks and other classifier strategies to identify the most relevant Stack Overflow tags. We then use these tags to implement collaborative filtering and recommend solutions based on similar existing posts in the database. The results displayed in this paper use Stack Overflow’s public dataset (https://www.kaggle. com/stackoverflow/stackoverflow).