Monitoring Valence with Facial EMG Sensors using Machine Learning

Affective Computing is an essential field of study with many impactful applications, from remote solutions for timely detection and improved management of mental health to improved human-computer interaction in the physical world and the metaverse. In this study, we explore the usage of a novel facial mask equipped with seven surface electromyography (sEMG) sensors to monitor subjective valence. We collected data from 30 participants who watched 20 affective videos (five exciting videos, five positive videos, five neutral videos, and five negative videos). The sensor data was filtered, and EMG features were extracted, including statistical and frequency-based features. The exploratory analysis indicated that the activations of the muscles associated with positive affect (left/right orbicularis and left/right zygomaticus) increased during positive periods compared to negative ones. Similarly, the activation of the corrugator muscle (related to negative affect) was increased during negative periods. This trend was also confirmed by the SHAP analysis performed on a machine-learning model for classifying 3-class valence (negative, neutral, and positive). On the test data of five unseen participants, the model achieved an accuracy of 63% and an F1-score of 49%. The class imbalance seemed to be one of the more significant challenges the ML models faced. The results confirmed the relationship between sEMG sensing and subjective valence but also showed that monitoring valence in real-time is challenging and requires more advanced approaches.