Skip to main content
SpringerLink
Log in

Multi-sensory Consistency Experience: A 6-DOF Simulation System Based on Video Automatically Generated Motion Effects

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14497))

Included in the following conference series:

  • 186 Accesses

Abstract

In this paper, we present a multi-sensory perception consistent 6-DOF motion system. The system automatically extracts the motion trajectory of the virtual camera as motion data from video and maps the motion data to the 6-DOF Stewart motion platform through a human perception-based wash-out algorithm and incorporates multi-sensory simulations of visual, auditory, tactile, and proprioceptive sensory perceptual consistency of the motion effect. The results of the user study showed that the system effectively enhanced the participants’ sense of realism and reduced the subjective perception of simulator discomfort. In addition, the system well supported users to self-create motion virtual environment through video, so that the public became the designer of motion experience content in the metaverse.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adel, A., et al.: Design of a 6-DOF hydraulic vehicle driving simulator. In: 2020 International Conference on Innovative Trends in Communication and Computer Engineering (ITCE), pp. 170–175. IEEE (2020)

    Google Scholar 

  2. Asadi, H., Bellmann, T., Mohamed, S., Lim, C.P., Khosravi, A., Nahavandi, S.: Adaptive motion cueing algorithm using optimized fuzzy control system for motion simulators. IEEE Trans. Intell. Veh. 8, 390–403 (2022)

    Google Scholar 

  3. Asadi, H., Lim, C.P., Mohamed, S., Nahavandi, D., Nahavandi, S.: Increasing motion fidelity in driving simulators using a fuzzy-based washout filter. IEEE Trans. Intell. Veh. 4(2), 298–308 (2019)

    Article  Google Scholar 

  4. Bimberg, P., Weissker, T., Kulik, A.: On the usage of the simulator sickness questionnaire for virtual reality research. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 464–467. IEEE (2020)

    Google Scholar 

  5. Buzan, D., Sclaroff, S., Kollios, G.: Extraction and clustering of motion trajectories in video. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol. 2, pp. 521–524. IEEE (2004)

    Google Scholar 

  6. Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M., Tardós, J.D.: ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021)

    Article  Google Scholar 

  7. Clifton, J., Palmisano, S.: Effects of steering locomotion and teleporting on cybersickness and presence in HMD-based virtual reality. Virtual Reality 24(3), 453–468 (2020)

    Article  Google Scholar 

  8. Dinh, H.Q., Walker, N., Hodges, L.F., Song, C., Kobayashi, A.: Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In: Proceedings of the IEEE Virtual Reality (Cat. No. 99CB36316), pp. 222–228. IEEE (1999)

    Google Scholar 

  9. Feng, M., Dey, A., Lindeman, R.W.: The effect of multi-sensory cues on performance and experience during walking in immersive virtual environments. In: 2016 IEEE Virtual Reality (VR), pp. 173–174. IEEE (2016)

    Google Scholar 

  10. Feng, M., Dey, A., Lindeman, R.W.: An initial exploration of a multi-sensory design space: tactile support for walking in immersive virtual environments. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI), pp. 95–104. IEEE (2016)

    Google Scholar 

  11. Hawkins, D.G.: Virtual reality and passive simulators: the future of fun. Commun. Age Virtual Reality 1, 159–89 (1995)

    Google Scholar 

  12. Kaliuzhna, M., Ferrè, E.R., Herbelin, B., Blanke, O., Haggard, P.: Multisensory effects on somatosensation: a trimodal visuo-vestibular-tactile interaction. Sci. Rep. 6(1), 26301 (2016)

    Article  Google Scholar 

  13. Khusro, Y.R., Zheng, Y., Grottoli, M., Shyrokau, B.: MPC-based motion-cueing algorithm for a 6-DOF driving simulator with actuator constraints. Vehicles 2(4), 625–647 (2020)

    Article  Google Scholar 

  14. Lee, J., Han, B., Choi, S.: Motion effects synthesis for 4d films. IEEE Trans. Vis. Comput. Graph. 22(10), 2300–2314 (2015)

    Article  Google Scholar 

  15. Melo, M., Gonçalves, G., Monteiro, P., Coelho, H., Vasconcelos-Raposo, J., Bessa, M.: Do multisensory stimuli benefit the virtual reality experience? A systematic review. IEEE Trans. Vis. Comput. Graph. 28(2), 1428–1442 (2020)

    Article  Google Scholar 

  16. Nehaoua, L., Mohellebi, H., Amouri, A., Arioui, H., Espié, S., Kheddar, A.: Design and control of a small-clearance driving simulator. IEEE Trans. Veh. Technol. 57(2), 736–746 (2008)

    Article  Google Scholar 

  17. Qazani, M.R.C., Asadi, H., Bellmann, T., Mohamed, S., Lim, C.P., Nahavandi, S.: Adaptive washout filter based on fuzzy logic for a motion simulation platform with consideration of joints’ limitations. IEEE Trans. Veh. Technol. 69(11), 12547–12558 (2020)

    Article  Google Scholar 

  18. Qazani, M.R.C., Asadi, H., Nahavandi, S.: An optimal motion cueing algorithm using the inverse kinematic solution of the hexapod simulation platform. IEEE Trans. Intell. Veh. 7(1), 73–82 (2021)

    Article  Google Scholar 

  19. Ranasinghe, N., et al.: Season traveller: multisensory narration for enhancing the virtual reality experience. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)

    Google Scholar 

  20. Rheinberg, F., Engeser, S., Vollmeyer, R.: Measuring components of flow: the flow-short-scale. In: Proceedings of the 1st International Positive Psychology Summit (2002)

    Google Scholar 

  21. Seo, S.M., Kimm, M.J.: Analysis of Virtual Reality Movies: Focusing on the Effect of Virtual Reality Movie’s Distinction on User Experience, pp. 308–312, July 2023. https://doi.org/10.1007/978-3-031-36004-6_42

  22. Sharma, A., Sharma, S., Chaudhary, M.: Are small travel agencies ready for digital marketing? Views of travel agency managers. Tour. Manag. 79, 104078 (2020)

    Google Scholar 

  23. Sheng, B., Li, P., Ali, R., Chen, C.L.P.: Improving video temporal consistency via broad learning system. IEEE Trans. Cybern. 52(7), 6662–6675 (2022). https://doi.org/10.1109/TCYB.2021.3079311

    Article  Google Scholar 

  24. Shin, S., Yoo, B., Han, S.: A framework for automatic creation of motion effects from theatrical motion pictures. Multimedia Syst. 20, 327–346 (2014)

    Article  Google Scholar 

  25. Stewart, D.: A platform with six degrees of freedom. Proc. Inst. Mech. Eng. 180(1), 371–386 (1965)

    Article  Google Scholar 

  26. Wang, Y., Sun, X., Shen, H., Yin, Y.: Research on improvement and optimization of washout algorithm for moving platform navigation simulator. In: 2021 IEEE 7th International Conference on Virtual Reality (ICVR), pp. 400–406. IEEE (2021)

    Google Scholar 

  27. Yang, T., Lai, I.K.W., Fan, Z.B., Mo, Q.M.: The impact of a 360 virtual tour on the reduction of psychological stress caused by COVID-19. Technol. Soc. 64, 101514 (2021)

    Article  Google Scholar 

  28. Yun, G., Lee, H., Han, S., Choi, S.: Improving viewing experiences of first-person shooter gameplays with automatically-generated motion effects. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2021)

    Google Scholar 

  29. Zachmann, G., Alcañiz Raya, M., Bourdot, P., Marchal, M., Stefanucci, J., Yang, X.: Correction to: virtual reality and mixed reality. In: Zachmann, G., Alcaniz Raya, M., Bourdot, P., Marchal, M., Stefanucci, J., Yang, X. (eds.) Virtual Reality and Mixed Reality. EuroXR 2022. LNCS, vol. 13484, p. C1. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-16234-3_14

  30. Zhu, J., et al.: Animating turbulent fluid with a robust and efficient high-order advection method. Comput. Animat. Virtual Worlds 31(4–5), e1951 (2020)

    Article  Google Scholar 

  31. Zou, X., et al.: On-road virtual reality autonomous vehicle (VRAV) simulator: an empirical study on user experience. Transp. Res. Part C Emerg. Technol. 126, 103090 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank all reviewers for their valuable comments. This work is supported by the National Natural Science Foundation of China under Grant (61972233, 62007021, 62277035).

Author information

Authors and Affiliations

  1. Shandong University, Ji’nan, China

    Hongqiu Luan, Yu Wang, Li Huang, Lutong Wang, Gaorong Lv, Wei Gai, Xiaona Luan & Chenglei Yang

Authors
  1. Hongqiu Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lutong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gaorong Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Gai
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaona Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chenglei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Gai or Xiaona Luan .

Editor information

Editors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  2. Shanghai Jiao Tong University, Shanghai, China

    Lei Bi

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. MIRALab-CUI, University of Geneva, Carouge, Geneve, Switzerland

    Nadia Magnenat-Thalmann

  5. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Luan, H. et al. (2024). Multi-sensory Consistency Experience: A 6-DOF Simulation System Based on Video Automatically Generated Motion Effects. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14497. Springer, Cham. https://doi.org/10.1007/978-3-031-50075-6_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50075-6_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50074-9

  • Online ISBN: 978-3-031-50075-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation