The effect of temporal asynchrony on audiovisual integration

Jingjing Yang, Xiujun Li, Qi Li, Jinglong Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many studies have shown that temporal asynchrony of visual-auditory stimuli can influence the integration between visual and auditory modality, however, the multisensory mechanisms of asynchronous inputs were not well understood. To investigate the effect of temporal asynchrony on audiovisual integration, we measured event-related potentials (ERPs) and behavior in human subjects. In this study, visual and auditory stimuli onset asynchronous (SOA= ±100 ms, ±50 ms, 0 ms), both the auditory and visual stimuli were attended. Using event-related potential, the differences of audiovisual integration elicited by asynchronous audiovisual stimuli were found at 140-200 ms over a wide parietal and occipital regions. These results revealed the asynchronous audiovisual stimuli enhanced the detection and elicited audiovisual integration. We expect that these results could helpful to process the audiovisual information of artificial intelligence.

Original languageEnglish
Title of host publication2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2338-2343
Number of pages6
ISBN (Electronic)9781509023943
DOIs
Publication statusPublished - Sep 1 2016
Event13th IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016 - Harbin, Heilongjiang, China
Duration: Aug 7 2016Aug 10 2016

Other

Other13th IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016
CountryChina
CityHarbin, Heilongjiang
Period8/7/168/10/16

Fingerprint

Service oriented architecture (SOA)
Artificial intelligence

Keywords

  • Audiovisual interaction
  • event-related potential
  • spatial information

ASJC Scopus subject areas

  • Mechanical Engineering
  • Artificial Intelligence
  • Computer Science Applications
  • Software

Cite this

Yang, J., Li, X., Li, Q., & Wu, J. (2016). The effect of temporal asynchrony on audiovisual integration. In 2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016 (pp. 2338-2343). [7558930] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMA.2016.7558930

The effect of temporal asynchrony on audiovisual integration. / Yang, Jingjing; Li, Xiujun; Li, Qi; Wu, Jinglong.

2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 2338-2343 7558930.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, J, Li, X, Li, Q & Wu, J 2016, The effect of temporal asynchrony on audiovisual integration. in 2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016., 7558930, Institute of Electrical and Electronics Engineers Inc., pp. 2338-2343, 13th IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016, Harbin, Heilongjiang, China, 8/7/16. https://doi.org/10.1109/ICMA.2016.7558930
Yang J, Li X, Li Q, Wu J. The effect of temporal asynchrony on audiovisual integration. In 2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 2338-2343. 7558930 https://doi.org/10.1109/ICMA.2016.7558930
Yang, Jingjing ; Li, Xiujun ; Li, Qi ; Wu, Jinglong. / The effect of temporal asynchrony on audiovisual integration. 2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 2338-2343
@inproceedings{93ddae3eedb54198aef39f009267f3a6,
title = "The effect of temporal asynchrony on audiovisual integration",
abstract = "Many studies have shown that temporal asynchrony of visual-auditory stimuli can influence the integration between visual and auditory modality, however, the multisensory mechanisms of asynchronous inputs were not well understood. To investigate the effect of temporal asynchrony on audiovisual integration, we measured event-related potentials (ERPs) and behavior in human subjects. In this study, visual and auditory stimuli onset asynchronous (SOA= ±100 ms, ±50 ms, 0 ms), both the auditory and visual stimuli were attended. Using event-related potential, the differences of audiovisual integration elicited by asynchronous audiovisual stimuli were found at 140-200 ms over a wide parietal and occipital regions. These results revealed the asynchronous audiovisual stimuli enhanced the detection and elicited audiovisual integration. We expect that these results could helpful to process the audiovisual information of artificial intelligence.",
keywords = "Audiovisual interaction, event-related potential, spatial information",
author = "Jingjing Yang and Xiujun Li and Qi Li and Jinglong Wu",
year = "2016",
month = "9",
day = "1",
doi = "10.1109/ICMA.2016.7558930",
language = "English",
pages = "2338--2343",
booktitle = "2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - The effect of temporal asynchrony on audiovisual integration

AU - Yang, Jingjing

AU - Li, Xiujun

AU - Li, Qi

AU - Wu, Jinglong

PY - 2016/9/1

Y1 - 2016/9/1

N2 - Many studies have shown that temporal asynchrony of visual-auditory stimuli can influence the integration between visual and auditory modality, however, the multisensory mechanisms of asynchronous inputs were not well understood. To investigate the effect of temporal asynchrony on audiovisual integration, we measured event-related potentials (ERPs) and behavior in human subjects. In this study, visual and auditory stimuli onset asynchronous (SOA= ±100 ms, ±50 ms, 0 ms), both the auditory and visual stimuli were attended. Using event-related potential, the differences of audiovisual integration elicited by asynchronous audiovisual stimuli were found at 140-200 ms over a wide parietal and occipital regions. These results revealed the asynchronous audiovisual stimuli enhanced the detection and elicited audiovisual integration. We expect that these results could helpful to process the audiovisual information of artificial intelligence.

AB - Many studies have shown that temporal asynchrony of visual-auditory stimuli can influence the integration between visual and auditory modality, however, the multisensory mechanisms of asynchronous inputs were not well understood. To investigate the effect of temporal asynchrony on audiovisual integration, we measured event-related potentials (ERPs) and behavior in human subjects. In this study, visual and auditory stimuli onset asynchronous (SOA= ±100 ms, ±50 ms, 0 ms), both the auditory and visual stimuli were attended. Using event-related potential, the differences of audiovisual integration elicited by asynchronous audiovisual stimuli were found at 140-200 ms over a wide parietal and occipital regions. These results revealed the asynchronous audiovisual stimuli enhanced the detection and elicited audiovisual integration. We expect that these results could helpful to process the audiovisual information of artificial intelligence.

KW - Audiovisual interaction

KW - event-related potential

KW - spatial information

UR - http://www.scopus.com/inward/record.url?scp=84991220895&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84991220895&partnerID=8YFLogxK

U2 - 10.1109/ICMA.2016.7558930

DO - 10.1109/ICMA.2016.7558930

M3 - Conference contribution

AN - SCOPUS:84991220895

SP - 2338

EP - 2343

BT - 2016 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -