Prediction of upcoming speech under fluent and disfluent conditions: eye tracking evidence from immersive virtual reality

Eleanor Huizeling*, David Peeters, Peter Hagoort

*Corresponding author for this work

    Research output: Contribution to journalArticleScientificpeer-review

    Abstract

    Traditional experiments indicate that prediction is important for efficient speech processing. In three virtual reality visual world paradigm experiments, we tested whether such findings hold in naturalistic settings (Experiment 1) and provided novel insights into whether disfluencies in speech (repairs/hesitations) inform one's predictions in rich environments (Experiments 2-3). Experiment 1 supports that listeners predict upcoming speech in naturalistic environments, with higher proportions of anticipatory target fixations in predictable compared to unpredictable trials. In Experiments 2-3, disfluencies reduced anticipatory fixations towards predicted referents, compared to conjunction (Experiment 2) and fluent (Experiment 3) sentences. Unexpectedly, Experiment 2 provided no evidence that participants made new predictions from a repaired verb. Experiment 3 provided novel findings that fixations towards the speaker increase upon hearing a hesitation, supporting current theories of how hesitations influence sentence processing. Together, these findings unpack listeners' use of visual (objects/speaker) and auditory (speech/disfluencies) information when predicting upcoming words.

    Original languageEnglish
    Pages (from-to)481-508
    Number of pages28
    JournalLanguage, Cognition and Neuroscience
    Volume37
    Issue number4
    Early online date30 Oct 2021
    DOIs
    Publication statusPublished - 2022

    Keywords

    • Prediction
    • Disfluencies
    • Visual world paradigm
    • Virtual reality
    • Eye tracking
    • SPOKEN WORD RECOGNITION
    • VISUAL-WORLD
    • TIME-COURSE
    • LANGUAGE COMPREHENSION
    • FILLED PAUSES
    • MOVEMENTS
    • HESITATIONS
    • INTEGRATION
    • ACTIVATION
    • ATTENTION

    Fingerprint

    Dive into the research topics of 'Prediction of upcoming speech under fluent and disfluent conditions: eye tracking evidence from immersive virtual reality'. Together they form a unique fingerprint.

    Cite this