Effect of pitch–space correspondence on sound-induced visual motion perception

Souta Hidaka, Wataru Teramoto, M.N. Keetels, J. Vroomen

Research output: Contribution to journalArticleScientificpeer-review

Abstract

The brain tends to associate specific features of stimuli across sensory modalities. The pitch of a sound is for example associated with spatial elevation such that higher-pitched sounds are felt as being “up” in space and lower-pitched sounds as being “down.” Here we investigated whether changes in the pitch of sounds could be effective for visual motion perception similar to those in the location of sounds. We demonstrated that only sounds that alternate in up/down location induced illusory vertical motion of a static visual stimulus, while sounds that alternate in higher/lower pitch did not induce this illusion. The pitch of a sound did not even modulate the visual motion perception induced by sounds alternating in up/down location. Interestingly, though, sounds alternating in higher/lower pitch could become a driver for visual motion if they were paired in a previous exposure phase with vertical visual apparent motion. Thus, only after prolonged exposure, the pitch of a sound became an inducer for upper/lower visual motion. This occurred even if during exposure the pitch and location of the sounds were paired in an incongruent fashion. These findings indicate that pitch–space correspondence is not so strong to drive or modulate visual motion perception. However, associative exposure could increase the saliency of pitch–space relationships and then the pitch could induce visual motion perception by itself.
Keywords: Crossmodal correspondence, Multisensory perception, Auditory space, Pitch, Visual motion perception
Original languageEnglish
Pages (from-to)117-126
JournalExperimental Brain Research
Volume231
Issue number1
DOIs
Publication statusPublished - 2013

Fingerprint

Motion Perception
Visual Perception

Keywords

  • Crossmodal correspondence
  • Multisensory perception
  • Auditory space
  • Pitch
  • Visual motion perception

Cite this

@article{ba7254c26584427bb954ee1ff86e26e1,
title = "Effect of pitch–space correspondence on sound-induced visual motion perception",
abstract = "The brain tends to associate specific features of stimuli across sensory modalities. The pitch of a sound is for example associated with spatial elevation such that higher-pitched sounds are felt as being “up” in space and lower-pitched sounds as being “down.” Here we investigated whether changes in the pitch of sounds could be effective for visual motion perception similar to those in the location of sounds. We demonstrated that only sounds that alternate in up/down location induced illusory vertical motion of a static visual stimulus, while sounds that alternate in higher/lower pitch did not induce this illusion. The pitch of a sound did not even modulate the visual motion perception induced by sounds alternating in up/down location. Interestingly, though, sounds alternating in higher/lower pitch could become a driver for visual motion if they were paired in a previous exposure phase with vertical visual apparent motion. Thus, only after prolonged exposure, the pitch of a sound became an inducer for upper/lower visual motion. This occurred even if during exposure the pitch and location of the sounds were paired in an incongruent fashion. These findings indicate that pitch–space correspondence is not so strong to drive or modulate visual motion perception. However, associative exposure could increase the saliency of pitch–space relationships and then the pitch could induce visual motion perception by itself.Keywords: Crossmodal correspondence, Multisensory perception, Auditory space, Pitch, Visual motion perception",
keywords = "Crossmodal correspondence, Multisensory perception, Auditory space, Pitch, Visual motion perception",
author = "Souta Hidaka and Wataru Teramoto and M.N. Keetels and J. Vroomen",
year = "2013",
doi = "10.1007/s00221-013-3674-2",
language = "English",
volume = "231",
pages = "117--126",
journal = "Experimental Brain Research",
issn = "0014-4819",
publisher = "Springer",
number = "1",

}

Effect of pitch–space correspondence on sound-induced visual motion perception. / Hidaka, Souta; Teramoto, Wataru; Keetels, M.N.; Vroomen, J.

In: Experimental Brain Research, Vol. 231, No. 1, 2013, p. 117-126.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Effect of pitch–space correspondence on sound-induced visual motion perception

AU - Hidaka, Souta

AU - Teramoto, Wataru

AU - Keetels, M.N.

AU - Vroomen, J.

PY - 2013

Y1 - 2013

N2 - The brain tends to associate specific features of stimuli across sensory modalities. The pitch of a sound is for example associated with spatial elevation such that higher-pitched sounds are felt as being “up” in space and lower-pitched sounds as being “down.” Here we investigated whether changes in the pitch of sounds could be effective for visual motion perception similar to those in the location of sounds. We demonstrated that only sounds that alternate in up/down location induced illusory vertical motion of a static visual stimulus, while sounds that alternate in higher/lower pitch did not induce this illusion. The pitch of a sound did not even modulate the visual motion perception induced by sounds alternating in up/down location. Interestingly, though, sounds alternating in higher/lower pitch could become a driver for visual motion if they were paired in a previous exposure phase with vertical visual apparent motion. Thus, only after prolonged exposure, the pitch of a sound became an inducer for upper/lower visual motion. This occurred even if during exposure the pitch and location of the sounds were paired in an incongruent fashion. These findings indicate that pitch–space correspondence is not so strong to drive or modulate visual motion perception. However, associative exposure could increase the saliency of pitch–space relationships and then the pitch could induce visual motion perception by itself.Keywords: Crossmodal correspondence, Multisensory perception, Auditory space, Pitch, Visual motion perception

AB - The brain tends to associate specific features of stimuli across sensory modalities. The pitch of a sound is for example associated with spatial elevation such that higher-pitched sounds are felt as being “up” in space and lower-pitched sounds as being “down.” Here we investigated whether changes in the pitch of sounds could be effective for visual motion perception similar to those in the location of sounds. We demonstrated that only sounds that alternate in up/down location induced illusory vertical motion of a static visual stimulus, while sounds that alternate in higher/lower pitch did not induce this illusion. The pitch of a sound did not even modulate the visual motion perception induced by sounds alternating in up/down location. Interestingly, though, sounds alternating in higher/lower pitch could become a driver for visual motion if they were paired in a previous exposure phase with vertical visual apparent motion. Thus, only after prolonged exposure, the pitch of a sound became an inducer for upper/lower visual motion. This occurred even if during exposure the pitch and location of the sounds were paired in an incongruent fashion. These findings indicate that pitch–space correspondence is not so strong to drive or modulate visual motion perception. However, associative exposure could increase the saliency of pitch–space relationships and then the pitch could induce visual motion perception by itself.Keywords: Crossmodal correspondence, Multisensory perception, Auditory space, Pitch, Visual motion perception

KW - Crossmodal correspondence

KW - Multisensory perception

KW - Auditory space

KW - Pitch

KW - Visual motion perception

U2 - 10.1007/s00221-013-3674-2

DO - 10.1007/s00221-013-3674-2

M3 - Article

VL - 231

SP - 117

EP - 126

JO - Experimental Brain Research

JF - Experimental Brain Research

SN - 0014-4819

IS - 1

ER -