Abstract for: LunaSim Copilot: An Integrated AI Assistant for System Dynamics Modeling

System dynamics (SD) modeling is the process of understanding and representing various elements of a complex system. While many SD modeling software facilitate this kind of process, the integration of an AI assistant into these software for expediting the creation and editing of these models is relatively underexplored. We developed LunaSim Copilot, a chat-based AI assistant that is integrated into our SD modeling software LunaSim. We evaluated four large language models (LLMs) as the models behind this AI assistant. These LLMs were tested on five tasks on generating SD models (increasing in difficulty) and were graded on rubrics we created. OpenAI’s o3-mini performed the best, with an average score of 94.6% (std. dev 8.4%). Claude 3.7 and Deepseek-R1 also had average scores over 90% (90.6% and 91.3%, respectively). In addition to accuracy, our evaluation rubrics included assessment of LLMs' ability to output with correct formatting and clear stock/flow/variable placement. High scores from LLMs suggest the practical usability of them as assistants in SD modeling. Developed AI assistant for SD modeling; NOT for writing manuscript.