The different effects of visual attention and auditory attention on bisensory audiovisual integration

Qi Li, Jingjing Yang, Jinglong Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Two experiments were conducted to explore the different effects of visual/auditory attention on bisensory audiovisual integration using behavioral and electrophysiological measures. Subjects were asked to attend visual stimulus in experiment1, but to attend auditory stimulus in experiment2. In each experiment, unisensory visual, unisensory auditory, and bisensory audiovisual stimuli were presented randomly with equal probability. The subjects were instructed to press a button in response to an infrequent target stimulus (20%) in the attend-designated modality. The behavioral results showed that the responses to audiovisual target stimuli were faster and more accurate than unisensory visual/auditory target stimuli no matter in experiment1 or in experiment2. These findings suggested that when presented simultaneously, visual and auditory stimuli interact to facilitate the behavioral response. The event-related potentials (ERPs) to bisensory stimuli (AV) were compared with the sum of the ERPs to unisensory auditory (A) and visual (V) stimuli. The bisensory audiovisual integration process was expressed as the difference waveform between AV and (A+V). In experiment1, three ERP components related to audiovisual integration were identified: (1) over the right temporal area, around 200-220 ms; (2) over the centro-medial area, around 290-310 ms; and (3) over the left and right ventral temporal area, around 290-310 ms. In experiment2, Two ERP components related to audiovisual integration were identified: (1) over the centro-medial area at 280-300 ms; and (2) over the right fronto-temporal area at 300-320 ms. Our results suggested that attention affects the process of the bisensory audiovisual integration.

Original languageEnglish
Title of host publication2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011
Pages328-333
Number of pages6
DOIs
Publication statusPublished - 2011
Event2011 5th IEEE/ICME International Conference on Complex Medical Engineering, CME 2011 - Harbin, China
Duration: May 22 2011May 25 2011

Other

Other2011 5th IEEE/ICME International Conference on Complex Medical Engineering, CME 2011
CountryChina
CityHarbin
Period5/22/115/25/11

Fingerprint

Experiments

Keywords

  • Attention
  • audiovisual integration
  • electroencephalography
  • event-related potential

ASJC Scopus subject areas

  • Biomedical Engineering

Cite this

Li, Q., Yang, J., & Wu, J. (2011). The different effects of visual attention and auditory attention on bisensory audiovisual integration. In 2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011 (pp. 328-333). [5876760] https://doi.org/10.1109/ICCME.2011.5876760

The different effects of visual attention and auditory attention on bisensory audiovisual integration. / Li, Qi; Yang, Jingjing; Wu, Jinglong.

2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011. 2011. p. 328-333 5876760.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, Q, Yang, J & Wu, J 2011, The different effects of visual attention and auditory attention on bisensory audiovisual integration. in 2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011., 5876760, pp. 328-333, 2011 5th IEEE/ICME International Conference on Complex Medical Engineering, CME 2011, Harbin, China, 5/22/11. https://doi.org/10.1109/ICCME.2011.5876760
Li Q, Yang J, Wu J. The different effects of visual attention and auditory attention on bisensory audiovisual integration. In 2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011. 2011. p. 328-333. 5876760 https://doi.org/10.1109/ICCME.2011.5876760
Li, Qi ; Yang, Jingjing ; Wu, Jinglong. / The different effects of visual attention and auditory attention on bisensory audiovisual integration. 2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011. 2011. pp. 328-333
@inproceedings{7ddf4b74e7b64471b84e44da2e271194,
title = "The different effects of visual attention and auditory attention on bisensory audiovisual integration",
abstract = "Two experiments were conducted to explore the different effects of visual/auditory attention on bisensory audiovisual integration using behavioral and electrophysiological measures. Subjects were asked to attend visual stimulus in experiment1, but to attend auditory stimulus in experiment2. In each experiment, unisensory visual, unisensory auditory, and bisensory audiovisual stimuli were presented randomly with equal probability. The subjects were instructed to press a button in response to an infrequent target stimulus (20{\%}) in the attend-designated modality. The behavioral results showed that the responses to audiovisual target stimuli were faster and more accurate than unisensory visual/auditory target stimuli no matter in experiment1 or in experiment2. These findings suggested that when presented simultaneously, visual and auditory stimuli interact to facilitate the behavioral response. The event-related potentials (ERPs) to bisensory stimuli (AV) were compared with the sum of the ERPs to unisensory auditory (A) and visual (V) stimuli. The bisensory audiovisual integration process was expressed as the difference waveform between AV and (A+V). In experiment1, three ERP components related to audiovisual integration were identified: (1) over the right temporal area, around 200-220 ms; (2) over the centro-medial area, around 290-310 ms; and (3) over the left and right ventral temporal area, around 290-310 ms. In experiment2, Two ERP components related to audiovisual integration were identified: (1) over the centro-medial area at 280-300 ms; and (2) over the right fronto-temporal area at 300-320 ms. Our results suggested that attention affects the process of the bisensory audiovisual integration.",
keywords = "Attention, audiovisual integration, electroencephalography, event-related potential",
author = "Qi Li and Jingjing Yang and Jinglong Wu",
year = "2011",
doi = "10.1109/ICCME.2011.5876760",
language = "English",
isbn = "9781424493241",
pages = "328--333",
booktitle = "2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011",

}

TY - GEN

T1 - The different effects of visual attention and auditory attention on bisensory audiovisual integration

AU - Li, Qi

AU - Yang, Jingjing

AU - Wu, Jinglong

PY - 2011

Y1 - 2011

N2 - Two experiments were conducted to explore the different effects of visual/auditory attention on bisensory audiovisual integration using behavioral and electrophysiological measures. Subjects were asked to attend visual stimulus in experiment1, but to attend auditory stimulus in experiment2. In each experiment, unisensory visual, unisensory auditory, and bisensory audiovisual stimuli were presented randomly with equal probability. The subjects were instructed to press a button in response to an infrequent target stimulus (20%) in the attend-designated modality. The behavioral results showed that the responses to audiovisual target stimuli were faster and more accurate than unisensory visual/auditory target stimuli no matter in experiment1 or in experiment2. These findings suggested that when presented simultaneously, visual and auditory stimuli interact to facilitate the behavioral response. The event-related potentials (ERPs) to bisensory stimuli (AV) were compared with the sum of the ERPs to unisensory auditory (A) and visual (V) stimuli. The bisensory audiovisual integration process was expressed as the difference waveform between AV and (A+V). In experiment1, three ERP components related to audiovisual integration were identified: (1) over the right temporal area, around 200-220 ms; (2) over the centro-medial area, around 290-310 ms; and (3) over the left and right ventral temporal area, around 290-310 ms. In experiment2, Two ERP components related to audiovisual integration were identified: (1) over the centro-medial area at 280-300 ms; and (2) over the right fronto-temporal area at 300-320 ms. Our results suggested that attention affects the process of the bisensory audiovisual integration.

AB - Two experiments were conducted to explore the different effects of visual/auditory attention on bisensory audiovisual integration using behavioral and electrophysiological measures. Subjects were asked to attend visual stimulus in experiment1, but to attend auditory stimulus in experiment2. In each experiment, unisensory visual, unisensory auditory, and bisensory audiovisual stimuli were presented randomly with equal probability. The subjects were instructed to press a button in response to an infrequent target stimulus (20%) in the attend-designated modality. The behavioral results showed that the responses to audiovisual target stimuli were faster and more accurate than unisensory visual/auditory target stimuli no matter in experiment1 or in experiment2. These findings suggested that when presented simultaneously, visual and auditory stimuli interact to facilitate the behavioral response. The event-related potentials (ERPs) to bisensory stimuli (AV) were compared with the sum of the ERPs to unisensory auditory (A) and visual (V) stimuli. The bisensory audiovisual integration process was expressed as the difference waveform between AV and (A+V). In experiment1, three ERP components related to audiovisual integration were identified: (1) over the right temporal area, around 200-220 ms; (2) over the centro-medial area, around 290-310 ms; and (3) over the left and right ventral temporal area, around 290-310 ms. In experiment2, Two ERP components related to audiovisual integration were identified: (1) over the centro-medial area at 280-300 ms; and (2) over the right fronto-temporal area at 300-320 ms. Our results suggested that attention affects the process of the bisensory audiovisual integration.

KW - Attention

KW - audiovisual integration

KW - electroencephalography

KW - event-related potential

UR - http://www.scopus.com/inward/record.url?scp=79959975109&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959975109&partnerID=8YFLogxK

U2 - 10.1109/ICCME.2011.5876760

DO - 10.1109/ICCME.2011.5876760

M3 - Conference contribution

AN - SCOPUS:79959975109

SN - 9781424493241

SP - 328

EP - 333

BT - 2011 IEEE/ICME International Conference on Complex Medical Engineering, CME 2011

ER -