TY - JOUR
T1 - The electrophysiology of lexical prediction of emoji and text
AU - Weissman, Benjamin
AU - Cohn, Neil
AU - Tanner, Darren
N1 - Copyright © 2024 Elsevier Ltd. All rights reserved.
PY - 2024/6/6
Y1 - 2024/6/6
N2 - As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two wellestablished ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
AB - As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two wellestablished ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
KW - Emoji
KW - Event-related potential (ERP)
KW - Late frontal positivity (LFP)
KW - Multimodality
KW - N400
KW - Prediction
UR - https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=wosstart_imp_pure20230417&SrcAuth=WosAPI&KeyUT=WOS:001224664700001&DestLinkType=FullRecord&DestApp=WOS_CPL
UR - http://www.scopus.com/inward/record.url?scp=85189484801&partnerID=8YFLogxK
U2 - 10.1016/j.neuropsychologia.2024.108881
DO - 10.1016/j.neuropsychologia.2024.108881
M3 - Article
C2 - 38579906
SN - 0028-3932
VL - 198
JO - Neuropsychologia
JF - Neuropsychologia
M1 - 108881
ER -