Gamification for data gathering in emergency response exercises

Kenny Meesters, Aaron Ruhe, Marvin Soetanto

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Our paper describes how gamification can be implemented in an emergency response exercise. In particular, we focus on the potential of gamification to support self-evaluation processes through the automated gathering of data about the participants' performance. Disaster-exercises are typically constructed around a scenario posing more or less known challenges to the participants, with the performance often determined only by observation. While these observations certainly have a value in their own right, since participants are deeply engaged and immersed in such exercises (e.g. detailed descriptions of events, immediate interpretation of data, and placing results in context) they also have their limitations. Especially in regards to learning in emergency response exercises we encounter limitations due to the observer's cognitive limitations and the (limited) type of data that is being gathered. While external observations may be sufficient for evaluation the execution of tasks according to predetermined standards, less well defined and complex processes, such as decision making processes, are more difficult to evaluate. Such processes are for example influenced by cognitive biases, group dynamics and even political motives. Finally, on a more practical level, the dependence on external observers adds an additional requirement to already resource-intense disaster exercises, especially considering the required expertise. We aim to demonstrate the potential of gamification in disaster exercises to support the structured collection of data regarding the performance of the participants. This data would support self-evaluation, in turn reducing the dependence on external observers and provide additional insights. First we derive different constructs from the various learning objectives associated with complex tasks such as group decision making, coordination and building situational awareness. Next we translate these constructs in quantifiable measurements enabling data collection. Finally, we implement these measurements in gamification elements in a disaster exercise game. The resulting data is presented to the participants enabling them to reflect on their own performance as a team reducing the dependence on external observers. This concept has been explored, developed and studied in an experimental setting. Ten volunteers were involved in the study, participating in a simulated emergency response exercise at the University of Amsterdam. In this test the focus of the exercise and the implemented gamifications elements were the decision making process, information management and coordination efforts. For validation purposes, the participants were divided over three different groups each given the same tasks and challenges. Additional several experts in disaster exercises and decision making were present to compare the outcome of the groups' self-evaluations with their notes as 'traditional' observers. Our results indicate that gamification can be a strong tool to gather quantitative data concerning the learning goals of the exercise that in turn helps participants to evaluate their own performance for complex tasks during disaster responses. Participants were to a large extend able to indicate the same lessons as the observers. Demonstrating that the collected data supported the participants to reflect on their performance and identify improvements themselves. Moreover, participants also indicated additional lessons learned which were not noted by the observers.
Original languageEnglish
Title of host publicationProceedings of the 9th European Conference on Games Based Learning (ECGBL 2015)
EditorsR. Munkvold, L. Kolås
Place of PublicationSteinkjer
PublisherAcademic Conferences and Publishing International Limited
Pages672-679
Number of pages1
ISBN (Print)9781910810583
Publication statusPublished - 2015
Event9th European Conference on Games Based Learning (ECGBL 2015) - Steinkjer, Norway
Duration: 8 Oct 20159 Oct 2015

Conference

Conference9th European Conference on Games Based Learning (ECGBL 2015)
CountryNorway
CitySteinkjer
Period8/10/159/10/15

Fingerprint

Disasters
Decision making
Information management

Cite this

Meesters, K., Ruhe, A., & Soetanto, M. (2015). Gamification for data gathering in emergency response exercises. In R. Munkvold, & L. Kolås (Eds.), Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015) (pp. 672-679). Steinkjer: Academic Conferences and Publishing International Limited.
Meesters, Kenny ; Ruhe, Aaron ; Soetanto, Marvin. / Gamification for data gathering in emergency response exercises. Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015) . editor / R. Munkvold ; L. Kolås. Steinkjer : Academic Conferences and Publishing International Limited, 2015. pp. 672-679
@inproceedings{213e4e34c2644c238a3dfc20df4eb54d,
title = "Gamification for data gathering in emergency response exercises",
abstract = "Our paper describes how gamification can be implemented in an emergency response exercise. In particular, we focus on the potential of gamification to support self-evaluation processes through the automated gathering of data about the participants' performance. Disaster-exercises are typically constructed around a scenario posing more or less known challenges to the participants, with the performance often determined only by observation. While these observations certainly have a value in their own right, since participants are deeply engaged and immersed in such exercises (e.g. detailed descriptions of events, immediate interpretation of data, and placing results in context) they also have their limitations. Especially in regards to learning in emergency response exercises we encounter limitations due to the observer's cognitive limitations and the (limited) type of data that is being gathered. While external observations may be sufficient for evaluation the execution of tasks according to predetermined standards, less well defined and complex processes, such as decision making processes, are more difficult to evaluate. Such processes are for example influenced by cognitive biases, group dynamics and even political motives. Finally, on a more practical level, the dependence on external observers adds an additional requirement to already resource-intense disaster exercises, especially considering the required expertise. We aim to demonstrate the potential of gamification in disaster exercises to support the structured collection of data regarding the performance of the participants. This data would support self-evaluation, in turn reducing the dependence on external observers and provide additional insights. First we derive different constructs from the various learning objectives associated with complex tasks such as group decision making, coordination and building situational awareness. Next we translate these constructs in quantifiable measurements enabling data collection. Finally, we implement these measurements in gamification elements in a disaster exercise game. The resulting data is presented to the participants enabling them to reflect on their own performance as a team reducing the dependence on external observers. This concept has been explored, developed and studied in an experimental setting. Ten volunteers were involved in the study, participating in a simulated emergency response exercise at the University of Amsterdam. In this test the focus of the exercise and the implemented gamifications elements were the decision making process, information management and coordination efforts. For validation purposes, the participants were divided over three different groups each given the same tasks and challenges. Additional several experts in disaster exercises and decision making were present to compare the outcome of the groups' self-evaluations with their notes as 'traditional' observers. Our results indicate that gamification can be a strong tool to gather quantitative data concerning the learning goals of the exercise that in turn helps participants to evaluate their own performance for complex tasks during disaster responses. Participants were to a large extend able to indicate the same lessons as the observers. Demonstrating that the collected data supported the participants to reflect on their performance and identify improvements themselves. Moreover, participants also indicated additional lessons learned which were not noted by the observers.",
author = "Kenny Meesters and Aaron Ruhe and Marvin Soetanto",
year = "2015",
language = "English",
isbn = "9781910810583",
pages = "672--679",
editor = "R. Munkvold and L. Kol{\aa}s",
booktitle = "Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015)",
publisher = "Academic Conferences and Publishing International Limited",

}

Meesters, K, Ruhe, A & Soetanto, M 2015, Gamification for data gathering in emergency response exercises. in R Munkvold & L Kolås (eds), Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015) . Academic Conferences and Publishing International Limited, Steinkjer, pp. 672-679, 9th European Conference on Games Based Learning (ECGBL 2015) , Steinkjer, Norway, 8/10/15.

Gamification for data gathering in emergency response exercises. / Meesters, Kenny; Ruhe, Aaron; Soetanto, Marvin.

Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015) . ed. / R. Munkvold; L. Kolås. Steinkjer : Academic Conferences and Publishing International Limited, 2015. p. 672-679.

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

TY - GEN

T1 - Gamification for data gathering in emergency response exercises

AU - Meesters, Kenny

AU - Ruhe, Aaron

AU - Soetanto, Marvin

PY - 2015

Y1 - 2015

N2 - Our paper describes how gamification can be implemented in an emergency response exercise. In particular, we focus on the potential of gamification to support self-evaluation processes through the automated gathering of data about the participants' performance. Disaster-exercises are typically constructed around a scenario posing more or less known challenges to the participants, with the performance often determined only by observation. While these observations certainly have a value in their own right, since participants are deeply engaged and immersed in such exercises (e.g. detailed descriptions of events, immediate interpretation of data, and placing results in context) they also have their limitations. Especially in regards to learning in emergency response exercises we encounter limitations due to the observer's cognitive limitations and the (limited) type of data that is being gathered. While external observations may be sufficient for evaluation the execution of tasks according to predetermined standards, less well defined and complex processes, such as decision making processes, are more difficult to evaluate. Such processes are for example influenced by cognitive biases, group dynamics and even political motives. Finally, on a more practical level, the dependence on external observers adds an additional requirement to already resource-intense disaster exercises, especially considering the required expertise. We aim to demonstrate the potential of gamification in disaster exercises to support the structured collection of data regarding the performance of the participants. This data would support self-evaluation, in turn reducing the dependence on external observers and provide additional insights. First we derive different constructs from the various learning objectives associated with complex tasks such as group decision making, coordination and building situational awareness. Next we translate these constructs in quantifiable measurements enabling data collection. Finally, we implement these measurements in gamification elements in a disaster exercise game. The resulting data is presented to the participants enabling them to reflect on their own performance as a team reducing the dependence on external observers. This concept has been explored, developed and studied in an experimental setting. Ten volunteers were involved in the study, participating in a simulated emergency response exercise at the University of Amsterdam. In this test the focus of the exercise and the implemented gamifications elements were the decision making process, information management and coordination efforts. For validation purposes, the participants were divided over three different groups each given the same tasks and challenges. Additional several experts in disaster exercises and decision making were present to compare the outcome of the groups' self-evaluations with their notes as 'traditional' observers. Our results indicate that gamification can be a strong tool to gather quantitative data concerning the learning goals of the exercise that in turn helps participants to evaluate their own performance for complex tasks during disaster responses. Participants were to a large extend able to indicate the same lessons as the observers. Demonstrating that the collected data supported the participants to reflect on their performance and identify improvements themselves. Moreover, participants also indicated additional lessons learned which were not noted by the observers.

AB - Our paper describes how gamification can be implemented in an emergency response exercise. In particular, we focus on the potential of gamification to support self-evaluation processes through the automated gathering of data about the participants' performance. Disaster-exercises are typically constructed around a scenario posing more or less known challenges to the participants, with the performance often determined only by observation. While these observations certainly have a value in their own right, since participants are deeply engaged and immersed in such exercises (e.g. detailed descriptions of events, immediate interpretation of data, and placing results in context) they also have their limitations. Especially in regards to learning in emergency response exercises we encounter limitations due to the observer's cognitive limitations and the (limited) type of data that is being gathered. While external observations may be sufficient for evaluation the execution of tasks according to predetermined standards, less well defined and complex processes, such as decision making processes, are more difficult to evaluate. Such processes are for example influenced by cognitive biases, group dynamics and even political motives. Finally, on a more practical level, the dependence on external observers adds an additional requirement to already resource-intense disaster exercises, especially considering the required expertise. We aim to demonstrate the potential of gamification in disaster exercises to support the structured collection of data regarding the performance of the participants. This data would support self-evaluation, in turn reducing the dependence on external observers and provide additional insights. First we derive different constructs from the various learning objectives associated with complex tasks such as group decision making, coordination and building situational awareness. Next we translate these constructs in quantifiable measurements enabling data collection. Finally, we implement these measurements in gamification elements in a disaster exercise game. The resulting data is presented to the participants enabling them to reflect on their own performance as a team reducing the dependence on external observers. This concept has been explored, developed and studied in an experimental setting. Ten volunteers were involved in the study, participating in a simulated emergency response exercise at the University of Amsterdam. In this test the focus of the exercise and the implemented gamifications elements were the decision making process, information management and coordination efforts. For validation purposes, the participants were divided over three different groups each given the same tasks and challenges. Additional several experts in disaster exercises and decision making were present to compare the outcome of the groups' self-evaluations with their notes as 'traditional' observers. Our results indicate that gamification can be a strong tool to gather quantitative data concerning the learning goals of the exercise that in turn helps participants to evaluate their own performance for complex tasks during disaster responses. Participants were to a large extend able to indicate the same lessons as the observers. Demonstrating that the collected data supported the participants to reflect on their performance and identify improvements themselves. Moreover, participants also indicated additional lessons learned which were not noted by the observers.

M3 - Conference contribution

SN - 9781910810583

SP - 672

EP - 679

BT - Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015)

A2 - Munkvold, R.

A2 - Kolås, L.

PB - Academic Conferences and Publishing International Limited

CY - Steinkjer

ER -

Meesters K, Ruhe A, Soetanto M. Gamification for data gathering in emergency response exercises. In Munkvold R, Kolås L, editors, Proceedings of the 9th European Conference on Games Based Learning (ECGBL 2015) . Steinkjer: Academic Conferences and Publishing International Limited. 2015. p. 672-679