Abstract
Multimodalities have attracted much interest in designing effective multimedia contents and services by applying sensory information. However, sensory experience from different modalities have been less understood in the multimedia contents design. Therefore, the optimal sensory experience is required to be considered to give impact on multimedia contents. This study is to determine the normalized perception for equalizing sensory difference between audio and visual modality. They were estimated by Just noticeable differences (JND) of audio and visual modalities and combination of both JNDs. JND for sound pressure and brightness were measured or estimated by curve fitting to calculate normalized just noticeable difference (NJND). NJND showed the intensity of sound pressure and brightness for the sensory balance and the sensory dominance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Klasen, M., Chen, Y.H., Mathiak, K.: Multisensory emotions: perception, combination and underlying neural processes. Rev. Neurosci. 23(4), 81–392 (2012)
Tabbers, H.K., Martens, R.L., Merriënboer, J.J.: Multimedia instructions and cognitive load theory: effects of modality and cueing. Br. J. Educ. Psychol. 74(1), 71–81 (2004)
Issa, N., Schuller, M., Santacaterina, S., Shapiro, M., Wang, E., Mayer, R.E., DaRosa, D.A.: Applying multimedia design principles enhances learning in medical education. Med. Educ. 45(8), 818–826 (2011)
Dubois, M., Vial, I.: Multimedia design: the effects of relating multimodal information. J. Comput. Assist. Learn. 16(2), 157–165 (2000)
Hecht, D., Reiner, M.: Sensory dominance in combinations of audio, visual and haptic stimuli. Exp. Brain Res. 193(2), 307–314 (2009)
Stevens, J.C., Marks, L.E.: Cross-modality matching of brightness and loudness. Proc. Natl. Acad. Sci. 54(2), 407–411 (1965)
Gescheider, G.A., Sklar, B.F., Van Doren, C.L., Verrillo, R.T.: Vibrotactile forward masking: psychophysical evidence for a triplex theory of cutaneous mechanoreception. J. Acoust. Soc. Am. 78(2), 534–543 (1985)
Väljamäe, A., Västfjäll, D., Larsson, P., Kleiner, M.: Perceived sound in mediated environments. In: Immersed in Media Experiences: Presence Psychology and Design (Handbook of Presence). Lawrence Erlbaum (2008, in print)
Acknowledgments
This work was partly supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No.2015-0-00312, The development of technology for social life logging based on analyzing social emotion and intelligence of convergence contents) and National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2011-0030079).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yoo, S., Whang, M. (2018). Multimedia Design Approaches by Just Noticeable Difference (JND) of Audiovisual Modalities. In: Park, J., Loia, V., Yi, G., Sung, Y. (eds) Advances in Computer Science and Ubiquitous Computing. CUTE CSA 2017 2017. Lecture Notes in Electrical Engineering, vol 474. Springer, Singapore. https://doi.org/10.1007/978-981-10-7605-3_210
Download citation
DOI: https://doi.org/10.1007/978-981-10-7605-3_210
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7604-6
Online ISBN: 978-981-10-7605-3
eBook Packages: EngineeringEngineering (R0)