The electrophysiology of lexical prediction of emoji and text

Benjamin Weissman, Neil Cohn, Darren Tanner

    Research output: Contribution to journalArticleScientificpeer-review

    1 Citation (Scopus)
    6 Downloads (Pure)

    Abstract

    As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two wellestablished ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
    Original languageEnglish
    Article number108881
    Number of pages11
    JournalNeuropsychologia
    Volume198
    Early online dateApr 2024
    DOIs
    Publication statusPublished - 6 Jun 2024

    Keywords

    • Emoji
    • Event-related potential (ERP)
    • Late frontal positivity (LFP)
    • Multimodality
    • N400
    • Prediction

    Fingerprint

    Dive into the research topics of 'The electrophysiology of lexical prediction of emoji and text'. Together they form a unique fingerprint.

    Cite this