Takayoshi Yamada
ABSTRACT
In order to construct a comfortable communication in the VR space, it is important to improve the speech comprehension in environmental noise. Although there have been many reports on the interaction between vision and acoustic, few studies using noisy VR spaces. In this study, sixteen Japanese male and female were tested to listen to a some sentence in a VR space with environmental noise, to evaluate the effect of the visual stimulus to the avatar speech comprehension against the environmental noise, with using the up-and-down method. The results showed that the cocktail party effect was also observed in the VR avatars, and the cocktail party effect continued even if the avatar vanished visually. In addition, it was suggested that the cocktail party effect was enhanced if the lip of the avatar synchronized correctly.
Supplemental Material
- Barry Arons. 1992. A review of the cocktail party effect. Journal of the American Voice I/O Society 12, 7 (1992), 35–50.Google Scholar
- Nigel Frangenberg, Kristoffer Waldow, and Arnulph Fuhrmann. 2021. Investigating the Influence of Sound Source Visualization on the Ventriloquism Effect in an Auralized Virtual Reality Environment. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 609–610.Google ScholarCross Ref
- Etienne Hendrickx, Mathieu Paquier, Vincent Koehl, and Julian Palacino. 2015. Ventriloquism effect with sound stimuli varying in both azimuth and elevation. The Journal of the Acoustical Society of America 138, 6 (2015), 3686–3697.Google ScholarCross Ref
- Charles E Jack and Willard R Thurlow. 1973. Effects of degree of visual association and angle of displacement on the “ventriloquism” effect. Perceptual and motor skills 37, 3 (1973), 967–979.Google Scholar
- Harry McGurk and John MacDonald. 1976. Hearing lips and seeing voices. Nature 264, 5588 (1976), 746–748.Google Scholar
- Sandra Poeschl, Konstantin Wall, and Nicola Doering. 2013. Integration of spatial sound in immersive virtual environments an experimental study on effects of spatial sound on presence. In 2013 IEEE Virtual Reality (VR). IEEE, 129–130.Google Scholar
- Atul Rungta, Nicholas Rewkowski, Carl Schissler, Philip Robinson, Ravish Mehra, and Dinesh Manocha. 2018. Effects of virtual acoustics on target-word identification performance in multi-talker environments. In Proceedings of the 15th ACM Symposium on Applied Perception. 1–8.Google ScholarDigital Library
- Unity Technologies. 2019. Unity DOCUMENTATION. https://docs.unity3d.com/ja/2019.4/Manual/class-AudioReverbFilter.html. [Online; accessed 2-Oct-2021].Google Scholar
Recommendations
Audio-visual speech recognition techniques in augmented reality environments
Many recent studies show that Augmented Reality (AR) and Automatic Speech Recognition (ASR) technologies can be used to help people with disabilities. Many of these studies have been performed only in their specialized field. Audio-Visual Speech ...
Hybrid avatars: enabling co-presence in multiple realities
Web3D '16: Proceedings of the 21st International Conference on Web3D TechnologyVirtual reality (VR) and augmented reality (AR) technologies are quickly making their way into people's everyday lives. Typically, these technologies are used separately to create either plain VR or AR applications rather than harnessing the ...
Walking up and down in immersive virtual worlds: Novel interactive techniques based on visual feedback
3DUI '10: Proceedings of the 2010 IEEE Symposium on 3D User InterfacesWe introduce novel interactive techniques to simulate the sensation of walking up and down in immersive virtual worlds based on visual feedback. Our method consists in modifying the motion of the virtual subjective camera while the user is really ...
Comments