The effect of spatial information of auditory stimuli on audiovisual interaction: Evidence from event-related potentials

Jingjing Yang, Tatsuya Ochi, Qi Li, Yulin Gao, Weiping Yang, Xiaoyu Tang, Satoshi Takahashi, Jinglong Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Converging evidence indicates that spatial information of stimuli plays an important role in multisensory interaction. The spatial characteristics of auditory stimuli tended to be biased toward the spatial characteristics of visual stimuli. However, when and where the multimodal audiovisual interaction is affected by the spatial characteristics of auditory stimuli remains unclear. This study used event-related potentials (ERPs) to demonstrate that the auditory stimuli come from 0°, 90°, 180° or 270° location affect the audiovisual interaction. The behavioral results showed that the responses to audiovisual target stimuli were faster than that to unimodal visual target stimuli. Moreover, the ERPs were recorded in response to unimodal auditory (A), unimodal visual (V) and bimodal (AV) stimuli. Cross-modal interactions were estimated using the additive [AV - A] model. At 0° location, four ERP components related to audiovisual interaction were observed: (1) Central and occipital effect was observed at around80 to 160ms; (2) over the central and occipital areas at around 160 to 200ms; (3) over the occipital areas at around 200 to 260ms. (4) Central and occipital areas after 350ms. These findings confirmed the main neural activity of audiovisual interaction. In addition, our study provided evidence that multimodal interaction can be generated even if the auditory stimuli come from different spatial locations.

Original languageEnglish
Title of host publication2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings
Pages699-704
Number of pages6
DOIs
Publication statusPublished - 2012
Event6th International Conference on Complex Medical Engineering, CME 2012 - Kobe, Japan
Duration: Jul 1 2012Jul 4 2012

Other

Other6th International Conference on Complex Medical Engineering, CME 2012
CountryJapan
CityKobe
Period7/1/127/4/12

Keywords

  • Audiovisual interaction
  • event-related potential
  • spatial information

ASJC Scopus subject areas

  • Biomedical Engineering

Cite this

Yang, J., Ochi, T., Li, Q., Gao, Y., Yang, W., Tang, X., ... Wu, J. (2012). The effect of spatial information of auditory stimuli on audiovisual interaction: Evidence from event-related potentials. In 2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings (pp. 699-704). [6275615] https://doi.org/10.1109/ICCME.2012.6275615

The effect of spatial information of auditory stimuli on audiovisual interaction : Evidence from event-related potentials. / Yang, Jingjing; Ochi, Tatsuya; Li, Qi; Gao, Yulin; Yang, Weiping; Tang, Xiaoyu; Takahashi, Satoshi; Wu, Jinglong.

2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings. 2012. p. 699-704 6275615.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, J, Ochi, T, Li, Q, Gao, Y, Yang, W, Tang, X, Takahashi, S & Wu, J 2012, The effect of spatial information of auditory stimuli on audiovisual interaction: Evidence from event-related potentials. in 2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings., 6275615, pp. 699-704, 6th International Conference on Complex Medical Engineering, CME 2012, Kobe, Japan, 7/1/12. https://doi.org/10.1109/ICCME.2012.6275615
Yang J, Ochi T, Li Q, Gao Y, Yang W, Tang X et al. The effect of spatial information of auditory stimuli on audiovisual interaction: Evidence from event-related potentials. In 2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings. 2012. p. 699-704. 6275615 https://doi.org/10.1109/ICCME.2012.6275615
Yang, Jingjing ; Ochi, Tatsuya ; Li, Qi ; Gao, Yulin ; Yang, Weiping ; Tang, Xiaoyu ; Takahashi, Satoshi ; Wu, Jinglong. / The effect of spatial information of auditory stimuli on audiovisual interaction : Evidence from event-related potentials. 2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings. 2012. pp. 699-704
@inproceedings{cc97390b727c4f8d8a2acf6dbe2f454d,
title = "The effect of spatial information of auditory stimuli on audiovisual interaction: Evidence from event-related potentials",
abstract = "Converging evidence indicates that spatial information of stimuli plays an important role in multisensory interaction. The spatial characteristics of auditory stimuli tended to be biased toward the spatial characteristics of visual stimuli. However, when and where the multimodal audiovisual interaction is affected by the spatial characteristics of auditory stimuli remains unclear. This study used event-related potentials (ERPs) to demonstrate that the auditory stimuli come from 0°, 90°, 180° or 270° location affect the audiovisual interaction. The behavioral results showed that the responses to audiovisual target stimuli were faster than that to unimodal visual target stimuli. Moreover, the ERPs were recorded in response to unimodal auditory (A), unimodal visual (V) and bimodal (AV) stimuli. Cross-modal interactions were estimated using the additive [AV - A] model. At 0° location, four ERP components related to audiovisual interaction were observed: (1) Central and occipital effect was observed at around80 to 160ms; (2) over the central and occipital areas at around 160 to 200ms; (3) over the occipital areas at around 200 to 260ms. (4) Central and occipital areas after 350ms. These findings confirmed the main neural activity of audiovisual interaction. In addition, our study provided evidence that multimodal interaction can be generated even if the auditory stimuli come from different spatial locations.",
keywords = "Audiovisual interaction, event-related potential, spatial information",
author = "Jingjing Yang and Tatsuya Ochi and Qi Li and Yulin Gao and Weiping Yang and Xiaoyu Tang and Satoshi Takahashi and Jinglong Wu",
year = "2012",
doi = "10.1109/ICCME.2012.6275615",
language = "English",
isbn = "9781467316163",
pages = "699--704",
booktitle = "2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings",

}

TY - GEN

T1 - The effect of spatial information of auditory stimuli on audiovisual interaction

T2 - Evidence from event-related potentials

AU - Yang, Jingjing

AU - Ochi, Tatsuya

AU - Li, Qi

AU - Gao, Yulin

AU - Yang, Weiping

AU - Tang, Xiaoyu

AU - Takahashi, Satoshi

AU - Wu, Jinglong

PY - 2012

Y1 - 2012

N2 - Converging evidence indicates that spatial information of stimuli plays an important role in multisensory interaction. The spatial characteristics of auditory stimuli tended to be biased toward the spatial characteristics of visual stimuli. However, when and where the multimodal audiovisual interaction is affected by the spatial characteristics of auditory stimuli remains unclear. This study used event-related potentials (ERPs) to demonstrate that the auditory stimuli come from 0°, 90°, 180° or 270° location affect the audiovisual interaction. The behavioral results showed that the responses to audiovisual target stimuli were faster than that to unimodal visual target stimuli. Moreover, the ERPs were recorded in response to unimodal auditory (A), unimodal visual (V) and bimodal (AV) stimuli. Cross-modal interactions were estimated using the additive [AV - A] model. At 0° location, four ERP components related to audiovisual interaction were observed: (1) Central and occipital effect was observed at around80 to 160ms; (2) over the central and occipital areas at around 160 to 200ms; (3) over the occipital areas at around 200 to 260ms. (4) Central and occipital areas after 350ms. These findings confirmed the main neural activity of audiovisual interaction. In addition, our study provided evidence that multimodal interaction can be generated even if the auditory stimuli come from different spatial locations.

AB - Converging evidence indicates that spatial information of stimuli plays an important role in multisensory interaction. The spatial characteristics of auditory stimuli tended to be biased toward the spatial characteristics of visual stimuli. However, when and where the multimodal audiovisual interaction is affected by the spatial characteristics of auditory stimuli remains unclear. This study used event-related potentials (ERPs) to demonstrate that the auditory stimuli come from 0°, 90°, 180° or 270° location affect the audiovisual interaction. The behavioral results showed that the responses to audiovisual target stimuli were faster than that to unimodal visual target stimuli. Moreover, the ERPs were recorded in response to unimodal auditory (A), unimodal visual (V) and bimodal (AV) stimuli. Cross-modal interactions were estimated using the additive [AV - A] model. At 0° location, four ERP components related to audiovisual interaction were observed: (1) Central and occipital effect was observed at around80 to 160ms; (2) over the central and occipital areas at around 160 to 200ms; (3) over the occipital areas at around 200 to 260ms. (4) Central and occipital areas after 350ms. These findings confirmed the main neural activity of audiovisual interaction. In addition, our study provided evidence that multimodal interaction can be generated even if the auditory stimuli come from different spatial locations.

KW - Audiovisual interaction

KW - event-related potential

KW - spatial information

UR - http://www.scopus.com/inward/record.url?scp=84867634901&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867634901&partnerID=8YFLogxK

U2 - 10.1109/ICCME.2012.6275615

DO - 10.1109/ICCME.2012.6275615

M3 - Conference contribution

AN - SCOPUS:84867634901

SN - 9781467316163

SP - 699

EP - 704

BT - 2012 ICME International Conference on Complex Medical Engineering, CME 2012 Proceedings

ER -