Natural Language Processing and Recommendation Engine for Stack Overflow Data
Name
Wang-jwlw2022-meng-eecs-2022-thesis.pdf
Description
Thesis PDF
Size
813.39 KB
Format
Adobe PDF
Checksum (MD5)
4c0e2c9e623415e2c579e95dee2422f9
Author(s)
Wang, Julia J.
Advisor(s)
Oliva, Aude
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
Query intent classification is important for information retrieval and problem solving. We use natural language processing and collaborative filtering algorithms to build a recommendation engine for Stack Overflow tag predictions. Our pipeline consists of document retrieval (TF-IDF and HOTT), text embedding (Sentence BERT), and classification (multi-label and multi-class). We experiment with neural networks and other classifier strategies to identify the most relevant Stack Overflow tags. We then use these tags to implement collaborative filtering and recommend solutions based on similar existing posts in the database. The results displayed in this paper use Stack Overflow’s public dataset (https://www.kaggle. com/stackoverflow/stackoverflow).
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link