Abstract for: Decision Heuristics and Human Performance in a Policy Management Simulation

Psychometric research has delivered reliable means for assessing various forms of intelligence, yet there has been relatively little success in predicting the human ability to solve complex problems in dynamic environments. The present work aims to profile dynamic decision making strategies using dynamic simulations in order to predict individual complex problem solving performance. We report an experiment assessing decision heuristics with the goal to predict complex problem solving ability. We used the COmplex DEcision Making (CODEM) system dynamics testbed to assess information seeking behaviors and the similarity of decision patterns to different types of heuristics. The Democracy 2 serious game is then used as an objective measure of complex problem solving ability. Democracy 2 is a realistic government management simulation requiring strong planning and systems thinking skills. A set of three new metrics is proposed to quantify similarity to different heuristics. Three models are compared on the basis of their predictive accuracy: a linear regression model, an artificial neural network and a support vector machine. Results show that the support vector machine has the most potential due to its superior results in a cross-validation test. We conclude with a discussion on future model extensions and generalization tests.