Can you handle this? The impact of object affordances on how co-speech gestures are produced

Ingrid Masson Carro, Martijn Goudbeek, Emiel Krahmer

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Hand gestures are tightly coupled with speech and with action. Hence, recent accounts have emphasised the idea that simulations of spatio-motoric imagery underlie the production of co-speech gestures. In this study, we suggest that action simulations directly influence the iconic strategies used by speakers to translate aspects of their mental representations into gesture. Using a classic referential paradigm, we investigate how speakers respond gesturally to the affordances of objects, by comparing the effects of describing objects that afford action performance (such as tools) and those that do not, on gesture production. Our results suggest that affordances play a key role in determining the amount of representational (but not non-representational) gestures produced by speakers, and the techniques chosen to depict such objects. To our knowledge, this is the first study to systematically show a connection between object characteristics and representation techniques in spontaneous gesture production during the depiction of static referents.
Original languageEnglish
JournalLanguage, Cognition and Neuroscience
Volume31
Issue number3
Early online date4 Nov 2015
Publication statusPublished - 15 Mar 2016

Fingerprint

Gestures
simulation
Imagery (Psychotherapy)
Affordances
Gesture
Co-speech Gestures
paradigm
performance
Simulation

Keywords

  • Gesture
  • action
  • representation techniques
  • simulation
  • affordances

Cite this

@article{eb15127d58924978843ebbd4f6291b31,
title = "Can you handle this?: The impact of object affordances on how co-speech gestures are produced",
abstract = "Hand gestures are tightly coupled with speech and with action. Hence, recent accounts have emphasised the idea that simulations of spatio-motoric imagery underlie the production of co-speech gestures. In this study, we suggest that action simulations directly influence the iconic strategies used by speakers to translate aspects of their mental representations into gesture. Using a classic referential paradigm, we investigate how speakers respond gesturally to the affordances of objects, by comparing the effects of describing objects that afford action performance (such as tools) and those that do not, on gesture production. Our results suggest that affordances play a key role in determining the amount of representational (but not non-representational) gestures produced by speakers, and the techniques chosen to depict such objects. To our knowledge, this is the first study to systematically show a connection between object characteristics and representation techniques in spontaneous gesture production during the depiction of static referents.",
keywords = "Gesture, action, representation techniques , simulation, affordances",
author = "{Masson Carro}, Ingrid and Martijn Goudbeek and Emiel Krahmer",
year = "2016",
month = "3",
day = "15",
language = "English",
volume = "31",
journal = "Language, Cognition and Neuroscience",
issn = "2327-3798",
publisher = "ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD",
number = "3",

}

Can you handle this? The impact of object affordances on how co-speech gestures are produced. / Masson Carro, Ingrid; Goudbeek, Martijn; Krahmer, Emiel.

In: Language, Cognition and Neuroscience, Vol. 31, No. 3, 15.03.2016.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Can you handle this?

T2 - The impact of object affordances on how co-speech gestures are produced

AU - Masson Carro, Ingrid

AU - Goudbeek, Martijn

AU - Krahmer, Emiel

PY - 2016/3/15

Y1 - 2016/3/15

N2 - Hand gestures are tightly coupled with speech and with action. Hence, recent accounts have emphasised the idea that simulations of spatio-motoric imagery underlie the production of co-speech gestures. In this study, we suggest that action simulations directly influence the iconic strategies used by speakers to translate aspects of their mental representations into gesture. Using a classic referential paradigm, we investigate how speakers respond gesturally to the affordances of objects, by comparing the effects of describing objects that afford action performance (such as tools) and those that do not, on gesture production. Our results suggest that affordances play a key role in determining the amount of representational (but not non-representational) gestures produced by speakers, and the techniques chosen to depict such objects. To our knowledge, this is the first study to systematically show a connection between object characteristics and representation techniques in spontaneous gesture production during the depiction of static referents.

AB - Hand gestures are tightly coupled with speech and with action. Hence, recent accounts have emphasised the idea that simulations of spatio-motoric imagery underlie the production of co-speech gestures. In this study, we suggest that action simulations directly influence the iconic strategies used by speakers to translate aspects of their mental representations into gesture. Using a classic referential paradigm, we investigate how speakers respond gesturally to the affordances of objects, by comparing the effects of describing objects that afford action performance (such as tools) and those that do not, on gesture production. Our results suggest that affordances play a key role in determining the amount of representational (but not non-representational) gestures produced by speakers, and the techniques chosen to depict such objects. To our knowledge, this is the first study to systematically show a connection between object characteristics and representation techniques in spontaneous gesture production during the depiction of static referents.

KW - Gesture

KW - action

KW - representation techniques

KW - simulation

KW - affordances

UR - http://dx.doi.org/10.1080/23273798.2015.1108448

M3 - Article

VL - 31

JO - Language, Cognition and Neuroscience

JF - Language, Cognition and Neuroscience

SN - 2327-3798

IS - 3

ER -