"Eliminating the Friction": An AI-powered Assistant for StarLogo Nova

Abstract

Agent-based modeling is a technique that allows students to reason about and create models of real-life phenomena. However, the programmatic implementations of this technique, such as StarLogo Nova, often introduce “friction”; students may get stuck on the syntactical details of the implementation before being able to engage in the mechanistic thinking behind their models. In order to shift students’ focus towards the goal of understanding the systems they are building, we set out to create an AI-powered assistant for StarLogo Nova that can explain and debug students’ code. After identifying and experimenting with various parameters of AI models in an attempt to improve their performance, we were able to build the StarLogo Turtle Helper, an easily accessible assistant integrated into the platform that can produce accurate responses to StarLogo-related questions. Through this process, we discovered two key properties of these models: first, the method through which these models use provided documentation (called retrieval-augmented generation, or RAG) is quite rudimentary, so any background knowledge should be included in the prompt or the model’s system instructions instead. Second, these models perform best if they are designed to only serve one purpose, so creating multiple models and chaining them together may be the best way to achieve more complex functionality.