Intrapersonal dependencies in multimodal behavior

Pieter A. Blomsma, Guido M. Linders, Julija Vaitonyte, Max M. Louwerse

Research output: Chapter in Book/Report/Conference proceedingChapterScientificpeer-review


Human interlocutors automatically adapt verbal and non-verbal signals so that different behaviors become synchronized over time. Multimodal communication comes naturally to humans, while this is not the case for Embodied Conversational Agents (ECAs). Knowing which behavioral channels synchronize within and across speakers and how they align seems critical in the development of ECAs. Yet, there exists little data-driven research that provides guidelines for the synchronization of different channels within an interlocutor. This study focuses on intrapersonal dependencies of multimodal behavior by using cross-recurrence analysis on a multimodal communication dataset to better understand the temporal relationships between language and gestural behavior channels. By shedding light on the intrapersonal synchronization of communicative channels in humans, we provide an initial manual for modality synchro-nisation in ECAs. CCS CONCEPTS · Human-centered computing → Empirical studies in HCI; · Computing methodologies → Discourse, dialogue and prag-matics.
Original languageEnglish
Title of host publicationIntrapersonal dependencies in multimodal behavior
PublisherAssociation for Computing Machinery (ACM)
Number of pages8
ISBN (Print)9781450375863
Publication statusPublished - 20 Oct 2020
EventIVA '20: ACM International Conference on Intelligent Virtual Agents - University of Glasgow, Glasgow, United Kingdom
Duration: 19 Oct 202023 Jan 2021
Conference number: 20


ConferenceIVA '20
Abbreviated titleIVA '20
Country/TerritoryUnited Kingdom
Internet address


Dive into the research topics of 'Intrapersonal dependencies in multimodal behavior'. Together they form a unique fingerprint.

Cite this