A behavioral and electrophysiological investigation of the effect of horizontal head viewing angle on audiovisual speech integration

We investigated how horizontal viewing angle of a speaking face influences audiovisual (AV) speech integration at the behavioral and neural level. In Experiment 1, seventeen participants identified consonant–vowel syllables (/faː/, /feː/, /foː/, /paː/, /peː/, /poː/, /taː/, /teː/, /toː/) presented in audiovisual, visual-only, and auditory-only conditions across four head orientations (frontal, oblique, profile, side-back at 0°, 45°, 90°, and 100°, respectively), in quiet and white-noise (–14 dB SNR) masking conditions. Audiovisual gain by lip read information (AV minus A-only accuracy) declined with increasing head rotations for speech-in-noise, but only for /p/, while accuracy in the visual-only condition followed a similar trend. Experiment 2 measured electrophysiology (EEG) in twenty-eight participants for four syllables (/faː/, /foː/, /paː/, /poː/) in quiet examining N1/P2 event-related potential (ERP) components for audiovisual, auditory-only, and visual-only stimulus presentations. Peak-amplitude and cluster-based analyses revealed that the well-documented N1 suppression by visual speech information (AV – V < A) was maximal at oblique (45°) head rotations, and significantly reduced at profile and side-back angles, whereas P2 suppression remained constant across all angles. N1 and P2 latencies were consistently shorter for AV – V than A-only conditions for all angles. These results demonstrate a dissociation between temporal (N1 amplitude suppression) and phonetic (P2 amplitude suppression) integrative mechanisms: temporal prediction degraded with profile/side-back head orientations, whereas audiovisual phonetic integration was not affected by the viewing angle of the head. Our findings indicate that head rotation reduces, but does not eliminate, audiovisual speech integration.