Gamified performance assessment of collaborative problem solving skills

Kristin Stoeffler*, Yigal Rosen, Maria Bolsinova, Alina A. von Davier

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review


In this paper we introduce a game-based approach for Collaborative Problem Solving (CPS) Skills assessment and provide preliminary evidence from a validation pilot study. To date, educational assessments have focused more heavily on the concrete, and accessible aspects of CPS with a diminished representation of the social aspects of CPS. We addressed this issue through the integration of our CPS construct into the game-based assessment "Circuit Runner" in which participants interact with a virtual agent to solve a series of challenges in a first-person maze environment (von Davier, 2017). Circuit Runner provides an environment that allows for controlled interdependence between a user and a virtual agent that facilitates the demonstration of the broad range of cognitive and social skills required for effective CPS. Tasks are designed to incorporate telemetry-based (e.g., log file, clickstream, interaction-based) and item response data to provide a more comprehensive measure of CPS skills. Our study included 500 participants on Amazon Mechanical Turk, who completed Circuit Runner, pre- and post-game surveys, and a CPS situational judgment test (CPS-SJT). These elements, in conjunction with the game-play, allowed for an expanded exploration of CPS skills with different modalities and types of instruments. The findings support and extend efforts to provide a stronger theoretical and empirical foundation for insights regarding CPS as a skillset, as well as the design of scalable game-based CPS assessments.

Original languageEnglish
Article number106036
Number of pages9
JournalComputers in Human Behavior
Publication statusPublished - 2020


  • Gamified assessment
  • Collaborative problem solving
  • Education
  • Human-agent collaboration
  • Chatbot
  • 21st century skills


Dive into the research topics of 'Gamified performance assessment of collaborative problem solving skills'. Together they form a unique fingerprint.

Cite this