Effects of ambiguous gestures and language on the time course of reference resolution

Max M Louwerse, Adrian Bangerter

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Two eye-tracking experiments investigated how and when pointing gestures and location descriptions affect target identification. The experiments investigated the effect of gestures and referring expressions on the time course of fixations to the target, using videos of human gestures and human voice, and animated gestures and synthesized speech. Ambiguous, yet informative pointing gestures elicited attention and facilitated target identification, akin to verbal location descriptions. Moreover, target identification was superior when both pointing gestures and verbal location descriptions were used. These findings suggest that gesture not only operates as a context to verbal descriptions, or that verbal descriptions operate as a context to gesture, but that they complement one another in reference resolution.

Original languageEnglish
Pages (from-to)1517-1529
Number of pages13
JournalCognitive Science
Volume34
Issue number8
DOIs
Publication statusPublished - Nov 2010
Externally publishedYes

Keywords

  • Gestures
  • Deixis
  • Deictic
  • Pointing
  • Multimodal communication
  • Referring expressions
  • Reference resolution
  • Eye tracking

Fingerprint Dive into the research topics of 'Effects of ambiguous gestures and language on the time course of reference resolution'. Together they form a unique fingerprint.

  • Cite this