Personalization of AI Tutor Based on Knowledge Graphs

Abstract

Personalized tutoring, tailored to the specific knowledge and needs of individual students, has been shown to significantly enhance academic performance. Research by Schmidt and Moust, for example, highlights that tutors who engage with students on a personal level are more effective in guiding them toward higher academic achievement [1]. Inspired by this principle, the Axiom group at the MIT Media Lab developed an AI tutor for their Intro to Programming courses. The initial version of the tutor, RAGS, relied on analyzing past conversations between students and the tutor, as well as course content, to generate personalized responses. While this approach showed promise, it faced scalability challenges, such as the need to store an ever-growing volume of conversation history and the risk of exceeding token limits in prompt context windows. Additionally, the model occasionally struggled with over-generalization, particularly when responding to vague questions based solely on historical interactions. To address these limitations, this thesis introduces a new approach: a student knowledge graph. Rather than relying on an expanding archive of past conversations, the knowledge graph uses weighted nodes to represent a student’s understanding of each concept. A weight of -8 indicates subpar understanding, while a weight of 8 signifies mastery. After pre-processing the course data, the graph maintains a fixed size, eliminating the need for additional storage over time. This innovation solves two critical problems: 1. Scalability: By leveraging a fixed-size PostgreSQL database, the student knowledge graph avoids the storage challenges associated with saving endless conversation histories. 2. Improved Personalization: Instead of sifting through old conversations, the tutor uses concept weights to generate more precise and contextually relevant responses, even to vague questions. Testing and evaluation of the implemented system demonstrate its effectiveness in both scalability and response quality. Over 60% of survey participants reported that the knowledge graph-enhanced tutor provided clearer and more relevant guidance, particularly when building on concepts they already understood. Additionally, over 80% of respondents noted improvements in the tutor’s ability to address weak areas and provide targeted practice, especially when preparing for quizzes or exams.