Effects of picture-word integration on reading visual narratives in L1 and L2

Multimodal education materials are pervasive in language learning. This study investigated the causal mechanisms of multimodal reading effects in first language (L1) and second language (L2). Seventy-five adult bilingual readers in Hong Kong read Chinese and English passages with different degrees of picture-word integration in a within-subject design. Results showed that tight text-picture integration facilitated better comprehension than independent text-picture presentation in L2, but not L1. Perceived ease and interest differentially mediated multimodal reading performance for L1 and L2 passages. Importantly, separate images in L2 passages led to poorer comprehension accuracy relative to plain text, but tended to have higher ratings of ease and interest, indicating that readers may be overconfident in their multimodal reading performance. In general, results support the notion that integration of text and pictures can moderate the process of meaning making, and these may differ depending on the language presented to a bilingual reader.