Systematic Evaluation of Design Choices for Deep Facial Action Coding Across Pose

Koichiro Niinuma, Itir Önal, Jeffrey Cohn, László Jeni

    Research output: Contribution to journalArticleScientificpeer-review

    Abstract

    The performance of automated facial expression coding is improving steadily. Advances
    in deep learning techniques have been key to this success. While the advantage of
    modern deep learning techniques is clear, the contribution of critical design choices
    remains largely unknown, especially for facial action unit occurrence and intensity across
    pose. Using the The Facial Expression Recognition and Analysis 2017 (FERA 2017)
    database, which provides a common protocol to evaluate robustness to pose variation,
    we systematically evaluated design choices in pre-training, feature alignment, model size
    selection, and optimizer details. Informed by the findings, we developed an architecture
    that exceeds state-of-the-art on FERA 2017. The architecture achieved a 3.5% increase
    in F1 score for occurrence detection and a 5.8% increase in Intraclass Correlation (ICC) for
    intensity estimation. To evaluate the generalizability of the architecture to unseen poses
    and new dataset domains, we performed experiments across pose in FERA 2017 and
    across domains in Denver Intensity of Spontaneous Facial Action (DISFA) and the UNBC
    Pain Archive.
    Original languageEnglish
    Article number636094
    Number of pages14
    JournalFrontiers in Computer Science
    Volume3
    DOIs
    Publication statusPublished - 29 Apr 2021

    Fingerprint

    Dive into the research topics of 'Systematic Evaluation of Design Choices for Deep Facial Action Coding Across Pose'. Together they form a unique fingerprint.

    Cite this