Generating Facial Expression Data: Computational and Experimental Evidence

Research output: Chapter in Book/Report/Conference proceedingChapterScientificpeer-review


It is crucial that naturally-looking Embodied Conversational Agents (ECAs) display various verbal and non-verbal behaviors, including facial expressions. The generation of credible facial expressions has been approached by means of different methods, yet remains difficult because of the availability of naturalistic data. To infuse more variability into the facial expressions of ECAs, we proposed a model that considered temporal dynamic of facial behaviors as a countable-state Markov process. Once trained, the model was able to output new sequences of facial expressions from an existing dataset containing facial videos with Action Unit (AU) encodings. The approach was validated by having computer software and humans identify facial emotion from video. Half of the videos employed newly generated sequences of facial expressions using the model while the other half contained sequences selected directly from the original dataset. We found no statistically significant evidence that the newly generated facial expression sequences could be differentiated from the original ones, demonstrating that the model was able to generate new facial expression data that were indistinguishable from the original data. Our proposed approach could be used to expand the amount of labelled facial expression data in order to create new training sets for machine learning methods.
Original languageEnglish
Title of host publicationIVA 2019 - Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents
PublisherAssociation for Computing Machinery, Inc
Number of pages3
ISBN (Print)9781450366724
Publication statusPublished - 1 Jul 2019
EventACM International Virtual Agents 2019 - Paris, France
Duration: 2 Jul 20195 Jul 2019

Publication series

NameIVA 2019 - Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents


ConferenceACM International Virtual Agents 2019
Abbreviated titleACM IVA 2019
Internet address


  • Embodied Conversational Agents
  • Facial Action Coding System (FACS)
  • Facial expressions
  • Machine Learning
  • Non-Verbal communication


Dive into the research topics of 'Generating Facial Expression Data: Computational and Experimental Evidence'. Together they form a unique fingerprint.

Cite this