Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life  Experts Playing The Sustainable Port

Gianluca Guglielmo; Michał Klincewicz; Elisabeth Huis in 't Veld; Pieter Spronck

doi:10.1007/978-3-031-78269-5_17

Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

50 Downloads (Pure)

Abstract

This study aimed to identify real-life experts working for a
port authority and lay people (students) who played The Sustainable
Port, a serious game aiming to simulate the dynamics occurring in a
port area. To achieve this goal, we analyzed eye gaze data collected non
invasively using low-grade webcams from 28 participants working for the
port authority of the Port of Rotterdam and 66 students. Such data were
used for a classification task implemented using a MiniRocket classifier,
an algorithm used for time-series classification. The classifier reached an
F1 score of 0.75 (SD = 0.07), a PR AUC of 0.73 (SD = 0.14), and an ROC
AUCof0.75 (SD = 0.15) providing evidence that it is possible to identify
real-life experts about maritime port management using data that can
be obtained from a webcam. We speculate that the gaze direction used
to train the MiniRocket may contain relevant information about the
cognitive processes and decisions occurring throughout the gameplay.
We suggest that the methods here presented not only can be used to
detect experts playing simulations, such as serious games, but also to
identify experts tackling screen-presented tasks.

Original language	English
Title of host publication	Games and Learning Alliance Conference (GALA)
Editors	Avo Schönbohm, Francesco Bellotti, Antonio Bucchiarone, Francesca de Rosa, Manuel Ninaus, Alf Wang, Vanissa Wanick, Pierpaolo Dondio
Publisher	Springer
Pages	177-187
Number of pages	11
DOIs	https://doi.org/10.1007/978-3-031-78269-5_17
Publication status	Published - 22 Nov 2024

Publication series

Name	Lecture Notes in Computer Science

Keywords

Gaze
Expertise
Machine Learning
Serious games
Maritime port
Time series classification

Access to Document

10.1007/978-3-031-78269-5_17

Detecting Experts Using a MiniRocketFinal published version, 198 KB

Cite this

Guglielmo, G., Klincewicz, M., Huis in 't Veld, E., & Spronck, P. (2024). Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port. In A. Schönbohm, F. Bellotti, A. Bucchiarone, F. de Rosa, M. Ninaus, A. Wang, V. Wanick, & P. Dondio (Eds.), Games and Learning Alliance Conference (GALA) (pp. 177-187). ( Lecture Notes in Computer Science). Springer. https://doi.org/10.1007/978-3-031-78269-5_17

Guglielmo, Gianluca ; Klincewicz, Michał ; Huis in 't Veld, Elisabeth et al. / Detecting Experts Using a MiniRocket : Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port. Games and Learning Alliance Conference (GALA). editor / Avo Schönbohm ; Francesco Bellotti ; Antonio Bucchiarone ; Francesca de Rosa ; Manuel Ninaus ; Alf Wang ; Vanissa Wanick ; Pierpaolo Dondio. Springer, 2024. pp. 177-187 ( Lecture Notes in Computer Science).

@inproceedings{91622b60ea634b938c9feba121d6806c,

title = "Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port",

abstract = "This study aimed to identify real-life experts working for a port authority and lay people (students) who played The Sustainable Port, a serious game aiming to simulate the dynamics occurring in a port area. To achieve this goal, we analyzed eye gaze data collected non invasively using low-grade webcams from 28 participants working for the port authority of the Port of Rotterdam and 66 students. Such data were used for a classification task implemented using a MiniRocket classifier, an algorithm used for time-series classification. The classifier reached an F1 score of 0.75 (SD = 0.07), a PR AUC of 0.73 (SD = 0.14), and an ROC AUCof0.75 (SD = 0.15) providing evidence that it is possible to identify real-life experts about maritime port management using data that can be obtained from a webcam. We speculate that the gaze direction used to train the MiniRocket may contain relevant information about the cognitive processes and decisions occurring throughout the gameplay. We suggest that the methods here presented not only can be used to detect experts playing simulations, such as serious games, but also to identify experts tackling screen-presented tasks.",

keywords = "Gaze, Expertise, Machine Learning, Serious games, Maritime port, Time series classification",

author = "Gianluca Guglielmo and Micha{\l} Klincewicz and \{Huis in 't Veld\}, Elisabeth and Pieter Spronck",

year = "2024",

month = nov,

day = "22",

doi = "10.1007/978-3-031-78269-5\_17",

language = "English",

series = " Lecture Notes in Computer Science",

publisher = "Springer",

pages = "177--187",

editor = "Sch{\"o}nbohm, \{Avo \} and Francesco Bellotti and Bucchiarone, \{Antonio \} and \{de Rosa\}, \{Francesca \} and Ninaus, \{Manuel \} and Wang, \{Alf \} and Wanick, \{Vanissa \} and Dondio, \{Pierpaolo \}",

booktitle = "Games and Learning Alliance Conference (GALA)",

address = "Germany",

}

Guglielmo, G , Klincewicz, M , Huis in 't Veld, E & Spronck, P 2024, Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port. in A Schönbohm, F Bellotti, A Bucchiarone, F de Rosa, M Ninaus, A Wang, V Wanick & P Dondio (eds), Games and Learning Alliance Conference (GALA). Lecture Notes in Computer Science, Springer, pp. 177-187. https://doi.org/10.1007/978-3-031-78269-5_17

Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port. / Guglielmo, Gianluca ; Klincewicz, Michał ; Huis in 't Veld, Elisabeth et al.
Games and Learning Alliance Conference (GALA). ed. / Avo Schönbohm; Francesco Bellotti; Antonio Bucchiarone; Francesca de Rosa; Manuel Ninaus; Alf Wang; Vanissa Wanick; Pierpaolo Dondio. Springer, 2024. p. 177-187 ( Lecture Notes in Computer Science).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Detecting Experts Using a MiniRocket

T2 - Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port

AU - Guglielmo, Gianluca

AU - Klincewicz, Michał

AU - Huis in 't Veld, Elisabeth

AU - Spronck, Pieter

PY - 2024/11/22

Y1 - 2024/11/22

N2 - This study aimed to identify real-life experts working for a port authority and lay people (students) who played The Sustainable Port, a serious game aiming to simulate the dynamics occurring in a port area. To achieve this goal, we analyzed eye gaze data collected non invasively using low-grade webcams from 28 participants working for the port authority of the Port of Rotterdam and 66 students. Such data were used for a classification task implemented using a MiniRocket classifier, an algorithm used for time-series classification. The classifier reached an F1 score of 0.75 (SD = 0.07), a PR AUC of 0.73 (SD = 0.14), and an ROC AUCof0.75 (SD = 0.15) providing evidence that it is possible to identify real-life experts about maritime port management using data that can be obtained from a webcam. We speculate that the gaze direction used to train the MiniRocket may contain relevant information about the cognitive processes and decisions occurring throughout the gameplay. We suggest that the methods here presented not only can be used to detect experts playing simulations, such as serious games, but also to identify experts tackling screen-presented tasks.

AB - This study aimed to identify real-life experts working for a port authority and lay people (students) who played The Sustainable Port, a serious game aiming to simulate the dynamics occurring in a port area. To achieve this goal, we analyzed eye gaze data collected non invasively using low-grade webcams from 28 participants working for the port authority of the Port of Rotterdam and 66 students. Such data were used for a classification task implemented using a MiniRocket classifier, an algorithm used for time-series classification. The classifier reached an F1 score of 0.75 (SD = 0.07), a PR AUC of 0.73 (SD = 0.14), and an ROC AUCof0.75 (SD = 0.15) providing evidence that it is possible to identify real-life experts about maritime port management using data that can be obtained from a webcam. We speculate that the gaze direction used to train the MiniRocket may contain relevant information about the cognitive processes and decisions occurring throughout the gameplay. We suggest that the methods here presented not only can be used to detect experts playing simulations, such as serious games, but also to identify experts tackling screen-presented tasks.

KW - Gaze

KW - Expertise

KW - Machine Learning

KW - Serious games

KW - Maritime port

KW - Time series classification

U2 - 10.1007/978-3-031-78269-5_17

DO - 10.1007/978-3-031-78269-5_17

M3 - Conference contribution

T3 - Lecture Notes in Computer Science

SP - 177

EP - 187

BT - Games and Learning Alliance Conference (GALA)

A2 - Schönbohm, Avo

A2 - Bellotti, Francesco

A2 - Bucchiarone, Antonio

A2 - de Rosa, Francesca

A2 - Ninaus, Manuel

A2 - Wang, Alf

A2 - Wanick, Vanissa

A2 - Dondio, Pierpaolo

PB - Springer

ER -

Guglielmo G , Klincewicz M , Huis in 't Veld E , Spronck P. Detecting Experts Using a MiniRocket: Gaze Direction Time Series Classification of Real-Life Experts Playing The Sustainable Port. In Schönbohm A, Bellotti F, Bucchiarone A, de Rosa F, Ninaus M, Wang A, Wanick V, Dondio P, editors, Games and Learning Alliance Conference (GALA). Springer. 2024. p. 177-187. ( Lecture Notes in Computer Science). doi: 10.1007/978-3-031-78269-5_17