What is your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing

Neil Cohn, Marta Kutas

    Research output: Contribution to journalArticleScientificpeer-review

    176 Downloads (Pure)

    Abstract

    Visual narratives sometimes depict successive images with different characters in the same physical space; corpus analysis has revealed that this occurs more often in Japanese manga than American comics. We used event-related brain potentials to determine whether comprehension of "visual narrative conjunctions" invokes not only incremental mental updating as traditionally assumed, but also, as we propose, "grammatical" combinatoric processing. We thus crossed (non)/conjunction sequences with character (in)/congruity. Conjunctions elicited a larger anterior negativity (300-500 ms) than nonconjunctions, regardless of congruity, implicating "grammatical" processes. Conjunction and incongruity both elicited larger P600s (500-700 ms), indexing updating. Both conjunction effects were modulated by participants' frequency of reading manga while growing up. Greater anterior negativity in frequent manga readers suggests more reliance on combinatoric processing; larger P600 effects in infrequent manga readers suggest more resources devoted to mental updating. As in language comprehension, it seems that processing conjunctions in visual narratives is not just mental updating but also partly grammatical, conditioned by comic readers' experience with specific visual narrative structures.

    Original languageEnglish
    Pages (from-to)27
    JournalCognitive research: principles and implications
    Volume2
    Issue number1
    DOIs
    Publication statusPublished - 2017

    Fingerprint

    Dive into the research topics of 'What is your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing'. Together they form a unique fingerprint.

    Cite this