Abstract
In this paper, we report on an experiment with The Walking Dead (TWD), which is a narrative-driven adventure game where players have to survive in a post-apocalyptic world filled with zombies. We used OpenFace software to extract action unit (AU) intensities of facial expressions characteristic of decision-making processes and then we implemented a simple convolution neural network (CNN) to see which AUs are predictive of decision-making. More specifically, this study aims
to identify the facial regions that are predictive of decision-making. Our results provide evidence that the pre-decision variations in action units 17 (chin raiser), 23 (lip tightener), and 25 (parting of lips) are predictive of decision-making processes. Furthermore, when combined, their predictive power increased up to .81 accuracy on the test set; we offer speculations about why it is that these particular three AUs were found to be connected to decision-making. Our results also suggest that machine learning methods in combination with video games may be used to accurately and automatically identify complex decision-making processes using AU intensity alone. Finally, our study offers a new method to test specific hypotheses about the relationships between higher-order cognitive processes and behavior, which relies on both narrative video games and easily accessible software, like OpenFace.
to identify the facial regions that are predictive of decision-making. Our results provide evidence that the pre-decision variations in action units 17 (chin raiser), 23 (lip tightener), and 25 (parting of lips) are predictive of decision-making processes. Furthermore, when combined, their predictive power increased up to .81 accuracy on the test set; we offer speculations about why it is that these particular three AUs were found to be connected to decision-making. Our results also suggest that machine learning methods in combination with video games may be used to accurately and automatically identify complex decision-making processes using AU intensity alone. Finally, our study offers a new method to test specific hypotheses about the relationships between higher-order cognitive processes and behavior, which relies on both narrative video games and easily accessible software, like OpenFace.
Original language | English |
---|---|
Title of host publication | Advances in Computer Games 2021 |
Subtitle of host publication | Lecture Notes in Computer Science |
Editors | C. Browne, A. Kishimoto, J. Schaeffer |
Publisher | Springer |
Pages | 187-196 |
Number of pages | 10 |
Volume | 13262 |
ISBN (Electronic) | 978-3-031-11488-5 |
ISBN (Print) | 978-3-031-11487-8 |
Publication status | Published - 2022 |
Keywords
- Video Games
- Decision-Making
- Facial Expression
- Facial Action Coding System (FACS)
- Machine Learning
- Deep Learning