The Role of the Auditory and Visual Modalities in the Perceptual Identification of Brazilian Portuguese Statements and Echo Questions

Luma Miranda*, Marc Swerts, Joao Moraes, Albert Rilliard

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)

Abstract

This paper presents the results of three perceptual experiments investigating the role of auditory and visual channels for the identification of statements and echo questions in Brazilian Portuguese. Ten Brazilian speakers (five male) were video-recorded (frontal view of the face) while they produced a sentence (“Como você sabe”), either as a statement (meaning “As you know.”) or as an echo question (meaning “As you know?”). Experiments were set up including the two different intonation contours. Stimuli were presented in conditions with clear and degraded audio as well as congruent and incongruent information from both channels. Results show that Brazilian listeners were able to distinguish statements and questions prosodically and visually, with auditory cues being dominant over visual ones. In noisy conditions, the visual channel improved the interpretation of prosodic cues robustly, while it degraded them in conditions where the visual information was incongruent with the auditory information. This study shows that auditory and visual information are integrated during speech perception, also when applied to prosodic patterns.

Original languageEnglish
Article number0023830919898886
Number of pages21
JournalLanguage and Speech
Early online date20 Jan 2020
DOIs
Publication statusE-pub ahead of print - 20 Jan 2020

Keywords

  • Audiovisual perception
  • Brazilian Portuguese
  • CUES
  • PROSODIC PROMINENCE
  • SPEECH
  • echo question
  • prosody
  • statement

Fingerprint Dive into the research topics of 'The Role of the Auditory and Visual Modalities in the Perceptual Identification of Brazilian Portuguese Statements and Echo Questions'. Together they form a unique fingerprint.

Cite this