Abstract for: A study of problem articulation in the system dynamics modeling process

Conceptualization is the most challenging process in system dynamics modeling. In response to this challenge, this study develops the Problem Articulation Process (PAP), a structured and systematic step-by-step approach to ensure consistency and clarity in problem articulation that is one of important process in conceptualization. This study conducts a comparative analysis using Large Language Models (LLMs) to evaluate PAP's effectiveness by comparing PAP and non-PAP cases. Metric is the similarity of the answers obtained from using PAP (PAP group) with the correct answers and those not using PAP (Non-PAP group) with the correct answers using BERTScore. The results indicate that PAP significantly enhances the quality of problem articulation. The use of PAP demonstrates that it effectively alleviates the challenges of problem articulation in conceptualization, thereby contributing to advancements in system dynamics modeling methodologies. The results of this comparative study have confirmed the effectiveness of PAP. The reason for this is that PAP is a gradual process that allows for intermediate reasoning, a characteristic of CoT. Non-PAP group, identifies six very low BERTScore and this study analyzes that this is “input conflicting hallucination.” PAP group does not have the hallucination because PAP acted as guidance and is able to suppress hallucinations. Asked LLM PAP questions