Effects of picture-word integration on reading visual narratives in L1 and L2

Yen Na Yum*, Neil Cohn, Way Kwok-Wai Lau

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Multimodal education materials are pervasive in language learning. This study investigated the causal mechanisms of multimodal reading effects in first language (L1) and second language (L2). Seventy-five adult bilingual readers in Hong Kong read Chinese and English passages with different degrees of picture-word integration in a within-subject design. Results showed that tight text-picture integration facilitated better comprehension than independent text-picture presentation in L2, but not L1. Perceived ease and interest differentially mediated multimodal reading performance for L1 and L2 passages. Importantly, separate images in L2 passages led to poorer comprehension accuracy relative to plain text, but tended to have higher ratings of ease and interest, indicating that readers may be overconfident in their multimodal reading performance. In general, results support the notion that integration of text and pictures can moderate the process of meaning making, and these may differ depending on the language presented to a bilingual reader.

Original languageEnglish
Article number101397
Number of pages11
JournalLearning and Instruction
Volume71
DOIs
Publication statusPublished - Feb 2021

Keywords

  • Bilingual
  • Multimodal reading
  • Perceived ease
  • Situational interest
  • Visual narratives
  • COMPREHENSION
  • CONSTRUCTION
  • ARCHITECTURE
  • CONTIGUITY
  • LEARNERS
  • MODALITY
  • MODELS
  • TEXT

Cite this