A Basic Psychophysics Study of Sound Reliability Effects on Audiovisual Integration for Developing New Virtual Reality Device

Hongtao Yu, Qiong Wu, Mengni Zhou, Qi Li, Jiajia Yang, Satoshi Takahashi, Yoshimichi Ejima, Jinglong Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Although visual virtual reality performance was widely investigated by researchers, it remains unclear that whether semantic reliability of sound can also facilitate the visual virtual reality performance. We investigated the behavioral category performance of living and non-living outlined drawings accompany with a semantic reliable or unreliable sound. We evaluate the living and non-living picture category performance under three multisensory semantic conditions: semantic reliable, semantic unreliable and semantic control (mix with semantic reliable and semantic unreliable stimuli). The reaction time results showed that faster category speed for semantic reliable multisensory presentation condition compared with semantic unreliable multisensory presentation condition whatever the outlined drawing was living or non-living. Such result indicated the multisensory integration facilitate the visual category performance. Additionally, non-living category has a significant advantage than living picture category under semantic unreliable multisensory condition, indicated nonliving objects had robust multisensory representation compared with living objects. This study provided potential theory support for developing new audiovisual virtual reality device or application.

Original languageEnglish
Title of host publication2021 IEEE International Conference on Mechatronics and Automation, ICMA 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1298-1303
Number of pages6
ISBN (Electronic)9781665441001
DOIs
Publication statusPublished - Aug 8 2021
Event18th IEEE International Conference on Mechatronics and Automation, ICMA 2021 - Takamatsu, Japan
Duration: Aug 8 2021Aug 11 2021

Publication series

Name2021 IEEE International Conference on Mechatronics and Automation, ICMA 2021

Conference

Conference18th IEEE International Conference on Mechatronics and Automation, ICMA 2021
Country/TerritoryJapan
CityTakamatsu
Period8/8/218/11/21

Keywords

  • multisensory integration
  • selective attention
  • semantic category
  • semantic reliability

ASJC Scopus subject areas

  • Artificial Intelligence
  • Electrical and Electronic Engineering
  • Mechanical Engineering
  • Control and Optimization

Fingerprint

Dive into the research topics of 'A Basic Psychophysics Study of Sound Reliability Effects on Audiovisual Integration for Developing New Virtual Reality Device'. Together they form a unique fingerprint.

Cite this