Abstract for: Calibration of Complex System Dynamics Models: A Practitioner's Report

A practitioner’s perspective on the calibration of complex system dynamics models is described in the context of a specific project in which a large, complex business training simulation model was converted from one language into I-Think® and design flaws in the original implementation were corrected. The model utilized 57 inputs and provided 296 outputs. The fact that calibration interacts with verification and validation is acknowledged, as well as the fact that the calibration strategy required for large models may not scale practically to smaller models. In additional to traditional best practices such as units checking, sensitivity testing, transient testing and graphical comparison, the paper focuses on a) simplifying and isolating interactions via submodels, using shims, slowing down feedback loops, creating cause and effect maps, testing at submodel level, and checking qualitative variables; b) Redesigning along the way; c) carefully documenting throughout the process; d) knowing when to step away; and e) building/acquiring automated tools. Having a calibration strategy for a large model is essential. The time required to apply the recommended methods can be significant, but the benefits clearly outweigh these costs. Nevertheless, even experienced modelers often wait too long before initiating the necessary disciplines.