Continuity, Co-reference, and Inference in Visual Sequencing

Bien Klomberg*, Irmak Hacımusaoğlu, Lenneke Lichtenberg, Joost Schilperoord, Neil Cohn

*Corresponding author for this work

    Research output: Contribution to journalArticleScientificpeer-review

    Abstract

    To understand narratives, comprehenders need to establish to what extent various expressions refer to the same entity, known as co-reference. Research in linguistics has long recognized this fundamental aspect of meaning-making, and perhaps as such, an increasing amount of works have attempted to extend this knowledge to other narrative expressions, such as visual storytelling. However, these applications of semantic and discourse models to visual narratives do not sufficiently account for how different visual signals may be reconciled into a common entity. This core question remains unanswered even in theories characterizing meaning relations for visual sequences in comics specifically. Therefore, this paper proposes a formal theory of visual co-reference that accounts for these issues. We present a model of constraints that describes visual co-reference based on varying correspondences between form (graphic input) and meaning (conceptual structure), including subsequent inferences about temporal and spatial relations across panels and more complex structures of visual discontinuity. This theory is in line with both corpus and psychological findings of visual narratives, and thus presents a robust framework for analyses of visual co-reference.
    Original languageEnglish
    Pages (from-to)1-43
    Number of pages43
    JournalGlossa: a journal of general linguistics
    Volume8
    Issue number1
    DOIs
    Publication statusPublished - 27 Sept 2023

    Keywords

    • co-reference
    • visual narratives
    • visual language
    • formal theory
    • conceptual semantics

    Fingerprint

    Dive into the research topics of 'Continuity, Co-reference, and Inference in Visual Sequencing'. Together they form a unique fingerprint.

    Cite this