Abstract for: Gaining Meta Understanding in Complex Situations. Development of an Evaluation Template

The ability to deal with complex situations is becoming increasingly important. There is evidence in literature that people have difficulties in understanding complex situations or problems appropriately, and various causes are suggested for this, such as reductionist thinking and the concept of bounded rationality. Systems thinking tools such as causal loops, stock and flow diagrams or simulations and dashboards are intended to counteract some of these problems and support people in exploring, understanding, anticipating, and managing complex situations. A certain meta-understanding is required to decide to what extent a tool is suitable for this and what understanding it conveys. That is, decision makers should understand how systemic and reliable the visualizations and models reflect the real situation and thus how systemic and reliable the understanding they provide is. In this paper, we propose an evaluation method for interactive tools to build this meta-understanding. Based on a literature review, we derived a list of tasks or questions and the corresponding information needs required to develop a systemic understanding. These served as the basis for the development of an assessment template.