Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants

M. Baart, J. Vroomen, K. Shaw, H. Bortfeld

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Infants and adults are well able to match auditory and visual speech, but the cues on which they rely (viz. temporal, phonetic and energetic correspondence in the auditory and visual speech streams) may differ. Here we assessed the relative contribution of the different cues using sine-wave speech (SWS). Adults (N = 52) and infants (N = 34, age ranged in between 5 and 15 months) matched 2 trisyllabic speech sounds (‘kalisu’ and ‘mufapi’), either natural or SWS, with visual speech information. On each trial, adults saw two articulating faces and matched a sound to one of these, while infants were presented the same stimuli in a preferential looking paradigm. Adults’ performance was almost flawless with natural speech, but was significantly less accurate with SWS. In contrast, infants matched the sound to the articulating face equally well for natural speech and SWS. These results suggest that infants rely to a lesser extent on phonetic cues than adults do to match audio to visual speech. This is in line with the notion that the ability to extract phonetic information from the visual signal increases during development, and suggests that phonetic knowledge might not be the basis for early audiovisual correspondence detection in speech.
Keywords: Audiovisual speech integration, Adults, Infants, Phonetic correspondence, Sine-wave speech
Original languageEnglish
Pages (from-to)31-43
JournalCognition
Volume130
Issue number1
DOIs
Publication statusPublished - 2014

Fingerprint

Phonetics
phonetics
infant
Cues
Audiovisual Speech
Waves

Cite this

@article{d18f44c6b4464d0590da8dd0170e7a52,
title = "Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants",
abstract = "Infants and adults are well able to match auditory and visual speech, but the cues on which they rely (viz. temporal, phonetic and energetic correspondence in the auditory and visual speech streams) may differ. Here we assessed the relative contribution of the different cues using sine-wave speech (SWS). Adults (N = 52) and infants (N = 34, age ranged in between 5 and 15 months) matched 2 trisyllabic speech sounds (‘kalisu’ and ‘mufapi’), either natural or SWS, with visual speech information. On each trial, adults saw two articulating faces and matched a sound to one of these, while infants were presented the same stimuli in a preferential looking paradigm. Adults’ performance was almost flawless with natural speech, but was significantly less accurate with SWS. In contrast, infants matched the sound to the articulating face equally well for natural speech and SWS. These results suggest that infants rely to a lesser extent on phonetic cues than adults do to match audio to visual speech. This is in line with the notion that the ability to extract phonetic information from the visual signal increases during development, and suggests that phonetic knowledge might not be the basis for early audiovisual correspondence detection in speech.Keywords: Audiovisual speech integration, Adults, Infants, Phonetic correspondence, Sine-wave speech",
author = "M. Baart and J. Vroomen and K. Shaw and H. Bortfeld",
year = "2014",
doi = "10.1016/j.cognition.2013.09.006",
language = "English",
volume = "130",
pages = "31--43",
journal = "Cognition",
issn = "0010-0277",
publisher = "Elsevier Science BV",
number = "1",

}

Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants. / Baart, M.; Vroomen, J.; Shaw, K.; Bortfeld, H.

In: Cognition, Vol. 130, No. 1, 2014, p. 31-43.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Degrading phonetic information affects matching of audiovisual speech in adults, but not in infants

AU - Baart, M.

AU - Vroomen, J.

AU - Shaw, K.

AU - Bortfeld, H.

PY - 2014

Y1 - 2014

N2 - Infants and adults are well able to match auditory and visual speech, but the cues on which they rely (viz. temporal, phonetic and energetic correspondence in the auditory and visual speech streams) may differ. Here we assessed the relative contribution of the different cues using sine-wave speech (SWS). Adults (N = 52) and infants (N = 34, age ranged in between 5 and 15 months) matched 2 trisyllabic speech sounds (‘kalisu’ and ‘mufapi’), either natural or SWS, with visual speech information. On each trial, adults saw two articulating faces and matched a sound to one of these, while infants were presented the same stimuli in a preferential looking paradigm. Adults’ performance was almost flawless with natural speech, but was significantly less accurate with SWS. In contrast, infants matched the sound to the articulating face equally well for natural speech and SWS. These results suggest that infants rely to a lesser extent on phonetic cues than adults do to match audio to visual speech. This is in line with the notion that the ability to extract phonetic information from the visual signal increases during development, and suggests that phonetic knowledge might not be the basis for early audiovisual correspondence detection in speech.Keywords: Audiovisual speech integration, Adults, Infants, Phonetic correspondence, Sine-wave speech

AB - Infants and adults are well able to match auditory and visual speech, but the cues on which they rely (viz. temporal, phonetic and energetic correspondence in the auditory and visual speech streams) may differ. Here we assessed the relative contribution of the different cues using sine-wave speech (SWS). Adults (N = 52) and infants (N = 34, age ranged in between 5 and 15 months) matched 2 trisyllabic speech sounds (‘kalisu’ and ‘mufapi’), either natural or SWS, with visual speech information. On each trial, adults saw two articulating faces and matched a sound to one of these, while infants were presented the same stimuli in a preferential looking paradigm. Adults’ performance was almost flawless with natural speech, but was significantly less accurate with SWS. In contrast, infants matched the sound to the articulating face equally well for natural speech and SWS. These results suggest that infants rely to a lesser extent on phonetic cues than adults do to match audio to visual speech. This is in line with the notion that the ability to extract phonetic information from the visual signal increases during development, and suggests that phonetic knowledge might not be the basis for early audiovisual correspondence detection in speech.Keywords: Audiovisual speech integration, Adults, Infants, Phonetic correspondence, Sine-wave speech

U2 - 10.1016/j.cognition.2013.09.006

DO - 10.1016/j.cognition.2013.09.006

M3 - Article

VL - 130

SP - 31

EP - 43

JO - Cognition

JF - Cognition

SN - 0010-0277

IS - 1

ER -