The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect

Research output: Contribution to journalArticleScientificpeer-review

126 Downloads (Pure)

Abstract

The Colavita effect refers to the phenomenon that when confronted with an audiovisual stimulus, observers report more often to have perceived the visual than the auditory component. The Colavita effect depends on low-level stimulus factors such as spatial and temporal proximity between the unimodal signals. Here, we examined whether the Colavita effect is modulated by synesthetic congruency between visual size and auditory pitch. If the Colavita effect depends on synesthetic congruency, we expect a larger Colavita effect for synesthetically congruent size/pitch (large visual stimulus/low-pitched tone; small visual stimulus/high-pitched tone) than synesthetically incongruent (large visual stimulus/high-pitched tone; small visual stimulus/low-pitched tone) combinations. Participants had to identify stimulus type (visual, auditory or audiovisual). The study replicated the Colavita effect because participants reported more often the visual than auditory component of the audiovisual stimuli. Synesthetic congruency had, however, no effect on the magnitude of the Colavita effect. EEG recordings to congruent and incongruent audiovisual pairings showed a late frontal congruency effect at 400–550 ms and an occipitoparietal effect at 690–800 ms with neural sources in the anterior cingulate and premotor cortex for the 400- to 550-ms window and premotor cortex, inferior parietal lobule and the posterior middle temporal gyrus for the 690- to 800-ms window. The electrophysiological data show that synesthetic congruency was probably detected in a processing stage subsequent to the Colavita effect. We conclude that—in a modality detection task—the Colavita effect can be modulated by low-level structural factors but not by higher-order associations between auditory and visual inputs.
Keywords
Synesthetic congruency Audiovisual integration Colavita effect Event-related potentials
Original languageEnglish
Pages (from-to)1209-1219
JournalExperimental Brain Research
Volume234
Issue number5
DOIs
Publication statusPublished - 2016

Fingerprint

Motor Cortex
Gyrus Cinguli
Electroencephalography

Cite this

@article{8bddded52bc041e59c649629d55a3ba6,
title = "The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect",
abstract = "The Colavita effect refers to the phenomenon that when confronted with an audiovisual stimulus, observers report more often to have perceived the visual than the auditory component. The Colavita effect depends on low-level stimulus factors such as spatial and temporal proximity between the unimodal signals. Here, we examined whether the Colavita effect is modulated by synesthetic congruency between visual size and auditory pitch. If the Colavita effect depends on synesthetic congruency, we expect a larger Colavita effect for synesthetically congruent size/pitch (large visual stimulus/low-pitched tone; small visual stimulus/high-pitched tone) than synesthetically incongruent (large visual stimulus/high-pitched tone; small visual stimulus/low-pitched tone) combinations. Participants had to identify stimulus type (visual, auditory or audiovisual). The study replicated the Colavita effect because participants reported more often the visual than auditory component of the audiovisual stimuli. Synesthetic congruency had, however, no effect on the magnitude of the Colavita effect. EEG recordings to congruent and incongruent audiovisual pairings showed a late frontal congruency effect at 400–550 ms and an occipitoparietal effect at 690–800 ms with neural sources in the anterior cingulate and premotor cortex for the 400- to 550-ms window and premotor cortex, inferior parietal lobule and the posterior middle temporal gyrus for the 690- to 800-ms window. The electrophysiological data show that synesthetic congruency was probably detected in a processing stage subsequent to the Colavita effect. We conclude that—in a modality detection task—the Colavita effect can be modulated by low-level structural factors but not by higher-order associations between auditory and visual inputs.KeywordsSynesthetic congruency Audiovisual integration Colavita effect Event-related potentials",
author = "J.J. Stekelenburg and M.N. Keetels",
year = "2016",
doi = "10.1007/s00221-015-4363-0",
language = "English",
volume = "234",
pages = "1209--1219",
journal = "Experimental Brain Research",
issn = "0014-4819",
publisher = "Springer",
number = "5",

}

The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect. / Stekelenburg, J.J.; Keetels, M.N.

In: Experimental Brain Research, Vol. 234, No. 5, 2016, p. 1209-1219.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The effect of synesthetic associations between the visual and auditory modalities on the Colavita effect

AU - Stekelenburg, J.J.

AU - Keetels, M.N.

PY - 2016

Y1 - 2016

N2 - The Colavita effect refers to the phenomenon that when confronted with an audiovisual stimulus, observers report more often to have perceived the visual than the auditory component. The Colavita effect depends on low-level stimulus factors such as spatial and temporal proximity between the unimodal signals. Here, we examined whether the Colavita effect is modulated by synesthetic congruency between visual size and auditory pitch. If the Colavita effect depends on synesthetic congruency, we expect a larger Colavita effect for synesthetically congruent size/pitch (large visual stimulus/low-pitched tone; small visual stimulus/high-pitched tone) than synesthetically incongruent (large visual stimulus/high-pitched tone; small visual stimulus/low-pitched tone) combinations. Participants had to identify stimulus type (visual, auditory or audiovisual). The study replicated the Colavita effect because participants reported more often the visual than auditory component of the audiovisual stimuli. Synesthetic congruency had, however, no effect on the magnitude of the Colavita effect. EEG recordings to congruent and incongruent audiovisual pairings showed a late frontal congruency effect at 400–550 ms and an occipitoparietal effect at 690–800 ms with neural sources in the anterior cingulate and premotor cortex for the 400- to 550-ms window and premotor cortex, inferior parietal lobule and the posterior middle temporal gyrus for the 690- to 800-ms window. The electrophysiological data show that synesthetic congruency was probably detected in a processing stage subsequent to the Colavita effect. We conclude that—in a modality detection task—the Colavita effect can be modulated by low-level structural factors but not by higher-order associations between auditory and visual inputs.KeywordsSynesthetic congruency Audiovisual integration Colavita effect Event-related potentials

AB - The Colavita effect refers to the phenomenon that when confronted with an audiovisual stimulus, observers report more often to have perceived the visual than the auditory component. The Colavita effect depends on low-level stimulus factors such as spatial and temporal proximity between the unimodal signals. Here, we examined whether the Colavita effect is modulated by synesthetic congruency between visual size and auditory pitch. If the Colavita effect depends on synesthetic congruency, we expect a larger Colavita effect for synesthetically congruent size/pitch (large visual stimulus/low-pitched tone; small visual stimulus/high-pitched tone) than synesthetically incongruent (large visual stimulus/high-pitched tone; small visual stimulus/low-pitched tone) combinations. Participants had to identify stimulus type (visual, auditory or audiovisual). The study replicated the Colavita effect because participants reported more often the visual than auditory component of the audiovisual stimuli. Synesthetic congruency had, however, no effect on the magnitude of the Colavita effect. EEG recordings to congruent and incongruent audiovisual pairings showed a late frontal congruency effect at 400–550 ms and an occipitoparietal effect at 690–800 ms with neural sources in the anterior cingulate and premotor cortex for the 400- to 550-ms window and premotor cortex, inferior parietal lobule and the posterior middle temporal gyrus for the 690- to 800-ms window. The electrophysiological data show that synesthetic congruency was probably detected in a processing stage subsequent to the Colavita effect. We conclude that—in a modality detection task—the Colavita effect can be modulated by low-level structural factors but not by higher-order associations between auditory and visual inputs.KeywordsSynesthetic congruency Audiovisual integration Colavita effect Event-related potentials

U2 - 10.1007/s00221-015-4363-0

DO - 10.1007/s00221-015-4363-0

M3 - Article

VL - 234

SP - 1209

EP - 1219

JO - Experimental Brain Research

JF - Experimental Brain Research

SN - 0014-4819

IS - 5

ER -