Do you see what they mean?

An eye-tracking study on the attention for gestures produced by people with aphasia

Kazuki Sekine, Karin van Nispen, Kim ten Felde, Jiska Koemans, Ellen van Drie, Basil Preisig

Research output: Contribution to conferencePosterOther research output

Abstract

Introduction People with aphasia (PWA) sometimes try to use alternative means of communication to convey their message. Studies have shown that they produce gestures when speaking, and, more importantly, these can convey information absent in their speech (Hogrefe et al., 2013; Mol et al., 2013). As such, gestures have the potential to contribute to the communication of PWA by helping interlocutors to understand their intended message. Studies on conversations by non-brain damaged participants (NBDP) have shown that interlocutors almost exclusively focus on the speaker’s face while speaking (Beattie et al., 2010; Preisig et al., 2015). However, it remains unclear whether healthy interlocutors do the same when conversing with PWA. Therefore, the present paper investigates up to what degree people pay attention to the gestures produced by PWA. Methods Participants Forty-six native speakers of Dutch (13 male) participated in our study. They were asked to watch videos of individuals (NBDP and PWA) explaining two scenarios. Participants were naïve to the purpose of this study. Design Stimuli were short videos of PWA (n=18) and NBDP (n=9) explaining two scenarios from the Scenario Test (van der Meulen et al., 2009). People in the videos explained one of two scenarios: a) buying a sweater, b) having witnessed an accident. The videos used were selected on the criteria that the participants used iconic gestures in their narrations. The videos lasted between 4 and 45 seconds. Participants watched all videos, of which the order was semi-randomized. Eye movements were recorded by means of a remote eye-tracker. Results We analyzed the time that participants looked at the gestures of NBDP and PWA. As the duration of the clips varied, we used proportions. An independent t-test showed that participants looked longer at gestures produced by PWA (M = 0.54, SD = 8.06) than at gestures produced by NBDP (M = 8.97, SD = 7.05), t(25) = -3.67, p = .001. Overall participants looked more at the face than at the gestures, but they looked less at the faces of PWA (M = 69.70, SD = 8.99) than at the faces of NBDP (M = 80.11, SD = 8.76), t(25) = 2.83, p < .01. Discussion Our study provided first evidence in the direction that interlocutors pay more attention to gestures of PWA than those of NBDP. This implies that the gestures produced by PWA are noted by the interlocutor and that they can pick up information conveyed by these gestures. Further research is desirable to determine if interlocutors also tend to understand the message better. At the conference we expect to present further analyses looking into the factors that guide people’s attention to gestures, such as: a) the attention for the gesture by the speakers themselves, but also, b) the comprehensibility of the spoken message.
Original languageEnglish
Pages76
Number of pages77
Publication statusPublished - 2018
EventInternational science of aphasia conference - Venice, Venice, Italy
Duration: 19 Sep 201822 Sep 2018
Conference number: 19

Conference

ConferenceInternational science of aphasia conference
Abbreviated titleSoA
CountryItaly
CityVenice
Period19/09/1822/09/18

Fingerprint

speech disorder
video
scenario
speaking
communication
narration
accident
conversation

Cite this

Sekine, K., van Nispen, K., ten Felde, K., Koemans, J., van Drie, E., & Preisig, B. (2018). Do you see what they mean? An eye-tracking study on the attention for gestures produced by people with aphasia. 76. Poster session presented at International science of aphasia conference, Venice, Italy.
Sekine, Kazuki ; van Nispen, Karin ; ten Felde, Kim ; Koemans, Jiska ; van Drie, Ellen ; Preisig, Basil. / Do you see what they mean? An eye-tracking study on the attention for gestures produced by people with aphasia. Poster session presented at International science of aphasia conference, Venice, Italy.77 p.
@conference{234f83772d7c4128b7a9df9165571238,
title = "Do you see what they mean?: An eye-tracking study on the attention for gestures produced by people with aphasia",
abstract = "Introduction People with aphasia (PWA) sometimes try to use alternative means of communication to convey their message. Studies have shown that they produce gestures when speaking, and, more importantly, these can convey information absent in their speech (Hogrefe et al., 2013; Mol et al., 2013). As such, gestures have the potential to contribute to the communication of PWA by helping interlocutors to understand their intended message. Studies on conversations by non-brain damaged participants (NBDP) have shown that interlocutors almost exclusively focus on the speaker’s face while speaking (Beattie et al., 2010; Preisig et al., 2015). However, it remains unclear whether healthy interlocutors do the same when conversing with PWA. Therefore, the present paper investigates up to what degree people pay attention to the gestures produced by PWA. Methods Participants Forty-six native speakers of Dutch (13 male) participated in our study. They were asked to watch videos of individuals (NBDP and PWA) explaining two scenarios. Participants were na{\"i}ve to the purpose of this study. Design Stimuli were short videos of PWA (n=18) and NBDP (n=9) explaining two scenarios from the Scenario Test (van der Meulen et al., 2009). People in the videos explained one of two scenarios: a) buying a sweater, b) having witnessed an accident. The videos used were selected on the criteria that the participants used iconic gestures in their narrations. The videos lasted between 4 and 45 seconds. Participants watched all videos, of which the order was semi-randomized. Eye movements were recorded by means of a remote eye-tracker. Results We analyzed the time that participants looked at the gestures of NBDP and PWA. As the duration of the clips varied, we used proportions. An independent t-test showed that participants looked longer at gestures produced by PWA (M = 0.54, SD = 8.06) than at gestures produced by NBDP (M = 8.97, SD = 7.05), t(25) = -3.67, p = .001. Overall participants looked more at the face than at the gestures, but they looked less at the faces of PWA (M = 69.70, SD = 8.99) than at the faces of NBDP (M = 80.11, SD = 8.76), t(25) = 2.83, p < .01. Discussion Our study provided first evidence in the direction that interlocutors pay more attention to gestures of PWA than those of NBDP. This implies that the gestures produced by PWA are noted by the interlocutor and that they can pick up information conveyed by these gestures. Further research is desirable to determine if interlocutors also tend to understand the message better. At the conference we expect to present further analyses looking into the factors that guide people’s attention to gestures, such as: a) the attention for the gesture by the speakers themselves, but also, b) the comprehensibility of the spoken message.",
author = "Kazuki Sekine and {van Nispen}, Karin and {ten Felde}, Kim and Jiska Koemans and {van Drie}, Ellen and Basil Preisig",
year = "2018",
language = "English",
pages = "76",
note = "International science of aphasia conference, SoA ; Conference date: 19-09-2018 Through 22-09-2018",

}

Sekine, K, van Nispen, K, ten Felde, K, Koemans, J, van Drie, E & Preisig, B 2018, 'Do you see what they mean? An eye-tracking study on the attention for gestures produced by people with aphasia' International science of aphasia conference, Venice, Italy, 19/09/18 - 22/09/18, pp. 76.

Do you see what they mean? An eye-tracking study on the attention for gestures produced by people with aphasia. / Sekine, Kazuki; van Nispen, Karin; ten Felde, Kim; Koemans, Jiska; van Drie, Ellen; Preisig, Basil.

2018. 76 Poster session presented at International science of aphasia conference, Venice, Italy.

Research output: Contribution to conferencePosterOther research output

TY - CONF

T1 - Do you see what they mean?

T2 - An eye-tracking study on the attention for gestures produced by people with aphasia

AU - Sekine, Kazuki

AU - van Nispen, Karin

AU - ten Felde, Kim

AU - Koemans, Jiska

AU - van Drie, Ellen

AU - Preisig, Basil

PY - 2018

Y1 - 2018

N2 - Introduction People with aphasia (PWA) sometimes try to use alternative means of communication to convey their message. Studies have shown that they produce gestures when speaking, and, more importantly, these can convey information absent in their speech (Hogrefe et al., 2013; Mol et al., 2013). As such, gestures have the potential to contribute to the communication of PWA by helping interlocutors to understand their intended message. Studies on conversations by non-brain damaged participants (NBDP) have shown that interlocutors almost exclusively focus on the speaker’s face while speaking (Beattie et al., 2010; Preisig et al., 2015). However, it remains unclear whether healthy interlocutors do the same when conversing with PWA. Therefore, the present paper investigates up to what degree people pay attention to the gestures produced by PWA. Methods Participants Forty-six native speakers of Dutch (13 male) participated in our study. They were asked to watch videos of individuals (NBDP and PWA) explaining two scenarios. Participants were naïve to the purpose of this study. Design Stimuli were short videos of PWA (n=18) and NBDP (n=9) explaining two scenarios from the Scenario Test (van der Meulen et al., 2009). People in the videos explained one of two scenarios: a) buying a sweater, b) having witnessed an accident. The videos used were selected on the criteria that the participants used iconic gestures in their narrations. The videos lasted between 4 and 45 seconds. Participants watched all videos, of which the order was semi-randomized. Eye movements were recorded by means of a remote eye-tracker. Results We analyzed the time that participants looked at the gestures of NBDP and PWA. As the duration of the clips varied, we used proportions. An independent t-test showed that participants looked longer at gestures produced by PWA (M = 0.54, SD = 8.06) than at gestures produced by NBDP (M = 8.97, SD = 7.05), t(25) = -3.67, p = .001. Overall participants looked more at the face than at the gestures, but they looked less at the faces of PWA (M = 69.70, SD = 8.99) than at the faces of NBDP (M = 80.11, SD = 8.76), t(25) = 2.83, p < .01. Discussion Our study provided first evidence in the direction that interlocutors pay more attention to gestures of PWA than those of NBDP. This implies that the gestures produced by PWA are noted by the interlocutor and that they can pick up information conveyed by these gestures. Further research is desirable to determine if interlocutors also tend to understand the message better. At the conference we expect to present further analyses looking into the factors that guide people’s attention to gestures, such as: a) the attention for the gesture by the speakers themselves, but also, b) the comprehensibility of the spoken message.

AB - Introduction People with aphasia (PWA) sometimes try to use alternative means of communication to convey their message. Studies have shown that they produce gestures when speaking, and, more importantly, these can convey information absent in their speech (Hogrefe et al., 2013; Mol et al., 2013). As such, gestures have the potential to contribute to the communication of PWA by helping interlocutors to understand their intended message. Studies on conversations by non-brain damaged participants (NBDP) have shown that interlocutors almost exclusively focus on the speaker’s face while speaking (Beattie et al., 2010; Preisig et al., 2015). However, it remains unclear whether healthy interlocutors do the same when conversing with PWA. Therefore, the present paper investigates up to what degree people pay attention to the gestures produced by PWA. Methods Participants Forty-six native speakers of Dutch (13 male) participated in our study. They were asked to watch videos of individuals (NBDP and PWA) explaining two scenarios. Participants were naïve to the purpose of this study. Design Stimuli were short videos of PWA (n=18) and NBDP (n=9) explaining two scenarios from the Scenario Test (van der Meulen et al., 2009). People in the videos explained one of two scenarios: a) buying a sweater, b) having witnessed an accident. The videos used were selected on the criteria that the participants used iconic gestures in their narrations. The videos lasted between 4 and 45 seconds. Participants watched all videos, of which the order was semi-randomized. Eye movements were recorded by means of a remote eye-tracker. Results We analyzed the time that participants looked at the gestures of NBDP and PWA. As the duration of the clips varied, we used proportions. An independent t-test showed that participants looked longer at gestures produced by PWA (M = 0.54, SD = 8.06) than at gestures produced by NBDP (M = 8.97, SD = 7.05), t(25) = -3.67, p = .001. Overall participants looked more at the face than at the gestures, but they looked less at the faces of PWA (M = 69.70, SD = 8.99) than at the faces of NBDP (M = 80.11, SD = 8.76), t(25) = 2.83, p < .01. Discussion Our study provided first evidence in the direction that interlocutors pay more attention to gestures of PWA than those of NBDP. This implies that the gestures produced by PWA are noted by the interlocutor and that they can pick up information conveyed by these gestures. Further research is desirable to determine if interlocutors also tend to understand the message better. At the conference we expect to present further analyses looking into the factors that guide people’s attention to gestures, such as: a) the attention for the gesture by the speakers themselves, but also, b) the comprehensibility of the spoken message.

M3 - Poster

SP - 76

ER -

Sekine K, van Nispen K, ten Felde K, Koemans J, van Drie E, Preisig B. Do you see what they mean? An eye-tracking study on the attention for gestures produced by people with aphasia. 2018. Poster session presented at International science of aphasia conference, Venice, Italy.