Gamified assessment of collaborative skills with chatbots

Kristin Stoeffler*, Yigal Rosen, Maria Bolsinova, Alina A. von Davier

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

5 Citations (Scopus)


Game-based assessments and learning environments create unique opportunities to provide learners with the ability to demonstrate their proficiency with cognitive skills and behaviors in increasingly authentic environments. Effective task designs, and the effective alignment of tasks with constructs, are also improving our ability to provide learners with insights about their proficiency with these skills. Sharing these insights within the industry with those working toward the same goal contributes to the rising tide that lifts all boats. In this paper we present insights from our work to develop and measure collaborative problem solving skills using a game-based assessment “Circuit Runner.” Our innovative educational game design allows us to incorporate item response data, telemetry data, and stealth- telemetry data to provide a more authentic measure of collaborative problem solving skills. Our study design included 379 study participants on Amazon Mechanical Turk (MTurk), who completed the “Circuit Runner” CPS assessment. The paper provides details on the design of educational games and scoring techniques and discusses findings from the pilot study.

Original languageEnglish
Title of host publicationArtificial Intelligence in Education - 19th International Conference, AIED 2018, Proceedings
EditorsRose Luckin, Kaska Porayska-Pomsta, Benedict du Boulay, Manolis Mavrikis, Carolyn Penstein Rosé, Bruce McLaren, Roberto Martinez-Maldonado, H. Ulrich Hoppe
PublisherSpringer Verlag
Number of pages5
ISBN (Print)9783319938455
Publication statusPublished - 2018
Externally publishedYes
Event19th International Conference on Artificial Intelligence in Education, AIED 2018 - London, United Kingdom
Duration: 27 Jun 201830 Jun 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10948 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference19th International Conference on Artificial Intelligence in Education, AIED 2018
Country/TerritoryUnited Kingdom


  • Assessment
  • Game
  • Stealth


Dive into the research topics of 'Gamified assessment of collaborative skills with chatbots'. Together they form a unique fingerprint.

Cite this