Skip to main content
Springer Nature Link
Log in

Augmented Virtuality–A Simplified, Scalable, and Modular Open-Source Unity Development System for Tangible VR with the Meta Quest 2

  • Conference paper
  • First Online:
Virtual, Augmented and Mixed Reality (HCII 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14707))

Included in the following conference series:

  • 641 Accesses

Abstract

With this article, we present the first – to our knowledge – development system for developing Augmented Virtuality (AV) applications with dynamic and tangible interfaces. In addition, we also propose a new taxonomy for differentiating between various types of AV applications.

The System: The system presented in this article, is developed for the Meta Quest 2 VR headset, and automatically handles the synchronization between virtual objects and their physical counterparts using the controller’s position. To ensure persistent real time alignment and synchronization during physical manipulations, it uses sensor feedback wirelessly transmitted from microcontrollers through UDP. In addition, the implemented user-interface for Unity enables users to use the system without the need for writing code. Furthermore, we have released the system as an open-source Unity assets package and invite others to use it for AV research-, educational-, industry-, or gaming purposes: https://github.com/BP-GITT/Tangible-VR/releases/tag/v1.0.0.

Taxonomy: The article also provides an overview of identified types, approaches, and methods for achieving Augmented Virtuality and propose a new practical orientated taxonomy for how to differentiate between them. The overview and taxonomy are a great place for AV application developers and new researchers to begin and includes the following six dimensions: 1) Type of AV, 2) approach to alignment, 3) synchronization method and sub-method, 4) number of synchronizable objects 5) the flexibility of their positioning and 6) their state.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://en.wikipedia.org/wiki/fiducial_marker.

  2. 2.

    https://unity.com/.

  3. 3.

    https://www.unrealengine.com/en-US.

  4. 4.

    https://www.vive.com/eu/accessory/tracker3/.

  5. 5.

    https://varjo.com/products/varjo-xr-3/.

  6. 6.

    https://www.meta.com/dk/en/quest/products/quest-2/.

  7. 7.

    https://docs.arduino.cc/hardware/uno-wifi-rev2/.

  8. 8.

    https://ardustore.dk/produkt/nodemcu-wemos-esp32-d1-mini-cp2104-udviklingsboard.

  9. 9.

    https://assetstore.unity.com/packages/tools/integration/oculus-integration-deprecated-82022.

  10. 10.

    https://www.arduino.cc/reference/en/libraries/wifinina/.

  11. 11.

    Military applications and purposes are exempt from this.

References

  1. Milgram, P., et al.: Augmented reality: a class of displays on the reality-virtuality continuum. In: Telemanipulator and Telepresence Technologies. SPIE (1995)

    Google Scholar 

  2. Murray, J.H.: Hamlet on the Holodeck. Updated edition: The Future of Narrative in Cyberspace. MIT Press (2017)

    Google Scholar 

  3. Pine, B.J., Korn, K.C.: Infinite Possibility: Creating Customer Value on the Digital Frontier. Berrett-Koehler Publishers (2011)

    Google Scholar 

  4. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. Roy. Soc. B Biol. Sci. 364(1535), 3549–3557 (2009)

    Article  Google Scholar 

  5. Waterworth, J., Riva, G.: Feeling Present in the Physical World and in Computer-Mediated Environments. Springer, London (2014). https://doi.org/10.1057/9781137431677

  6. Lombard, M., Ditton, T.: At the heart of it all: the concept of presence. J. Comput.-Mediated Commun. 3(2), JCMC321 (1997)

    Google Scholar 

  7. Slater, M., Wilbur, S.: A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence Teleoperators Virtual Environ. 6(6), 603–616 (1997)

    Google Scholar 

  8. Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence 7(3), 225–240 (1998)

    Article  Google Scholar 

  9. Schrader, C.: The relation between virtual presence and learning outcomes in serious games-the mediating effect of motivation. IxD&A 19, 38–46 (2013)

    Article  Google Scholar 

  10. Rupp, M.A., et al.: Investigating learning outcomes and subjective experiences in 360-degree videos. Comput. Educ. 128, 256–268 (2019)

    Article  Google Scholar 

  11. Elmezeny, A., Edenhofer, N., Wimmer, J.: Immersive storytelling in 360-degree videos: an analysis of interplay between narrative and technical immersion. J. Virtual Worlds Res. 11(1) (2018)

    Google Scholar 

  12. Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (1997)

    Google Scholar 

  13. Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Syst. J. 39(3.4), 915–931 (2000)

    Google Scholar 

  14. Hinckley, K., et al.: Passive real-world interface props for neurosurgical visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (1994)

    Google Scholar 

  15. Insko, B.E.: Passive haptics significantly enhances virtual environments. The University of North Carolina at Chapel Hill (2001)

    Google Scholar 

  16. Araujo, B., et al.: Snake charmer: physically enabling virtual objects. In: Proceedings of the TEI 2016: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (2016)

    Google Scholar 

  17. Dourish, P.: Where the Action Is: the Foundations of Embodied Interaction. MIT Press (2001)

    Google Scholar 

  18. Johnson, M.P., et al.: Sympathetic interfaces: using a plush toy to direct synthetic characters. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (1999)

    Google Scholar 

  19. Uhl, J.C., et al.: Tangible immersive trauma simulation: is mixed reality the next level of medical skills training? In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023)

    Google Scholar 

  20. Baek, S., Gil, Y.-H., Kim, Y.: VR-based job training system using tangible interactions. Sensors 21(20), 6794 (2021)

    Article  Google Scholar 

  21. Van Campenhout, L., Van Camp, M., Vancoppenolle, W.: Exploring tangible VR as a tool for workplace design. In: Companion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces (2020)

    Google Scholar 

  22. Palma, G., Perry, S., Cignoni, P.: Augmented virtuality using touch-sensitive 3D-printed objects. Remote Sens. 13(11), 2186 (2021)

    Article  Google Scholar 

  23. Mercado, V.R., et al.: Alfred: the haptic butler on-demand tangibles for object manipulation in virtual reality using an ETHD. In: 2021 IEEE World Haptics Conference (WHC). IEEE (2021)

    Google Scholar 

  24. Matviienko, A., et al.: VRtangibles: assisting children in creating virtual scenes using tangible objects and touch input. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (2021)

    Google Scholar 

  25. Cardoso, J.C., Ribeiro, J.M.: Tangible VR book: exploring the design space of marker-based tangible interfaces for virtual reality. Appl. Sci. 11(4), 1367 (2021)

    Article  Google Scholar 

  26. Morikubo, Y., et al.: Tangible projection mapping: Dynamic appearance augmenting of objects in hands. In: SIGGRAPH Asia 2018 Emerging Technologies, pp. 1–2 (2018)

    Google Scholar 

  27. Nomoto, T., Koishihara, R., Watanabe, Y.: Realistic dynamic projection mapping using real-time ray tracing. In: ACM SIGGRAPH 2020 Emerging Technologies, pp. 1–2 (2020)

    Google Scholar 

  28. Harley, D., et al.: Tangible VR: diegetic tangible objects for virtual reality narratives. In: Proceedings of the 2017 Conference on Designing Interactive Systems (2017)

    Google Scholar 

  29. Valentini, I., et al.: Improving obstacle awareness to enhance interaction in virtual reality. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE (2020)

    Google Scholar 

  30. Lou, W.J., Teh, A.S., Phang, S.K.: Low cost haptic and motion based mixed reality peripheral interface. In: AIP Conference Proceedings. AIP Publishing (2019)

    Google Scholar 

  31. Fang, C., et al.: Wireality: enabling complex tangible geometries in virtual reality with worn multi-string haptics. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020)

    Google Scholar 

  32. Teng, S.-Y., et al.: TilePoP: tile-type pop-up prop for virtual reality. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (2019)

    Google Scholar 

  33. VR.Space: Which Popular VR Headsets Have Generated the Most Revenue? (2023). https://vr.space/news/equipment/which-popular-vr-headsets-have-generated-the-most-revenue/

Download references

Author information

Authors and Affiliations

  1. Maersk Mc-Kinney Moller Institute, Game Development and Learning Technologies, University of Southern Denmark, Campusvej 55, 5230, Odense M, Denmark

    Bjarke Kristian Maigaard Kjær Pedersen, Patricia Bianca Lyk & Daniel Alexander Auerbach

Authors
  1. Bjarke Kristian Maigaard Kjær Pedersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia Bianca Lyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Alexander Auerbach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bjarke Kristian Maigaard Kjær Pedersen .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Adelphi, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities, Orlando, FL, USA

    Gino Fragomeni

Ethics declarations

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pedersen, B.K.M.K., Lyk, P.B., Alexander Auerbach, D. (2024). Augmented Virtuality–A Simplified, Scalable, and Modular Open-Source Unity Development System for Tangible VR with the Meta Quest 2. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. HCII 2024. Lecture Notes in Computer Science, vol 14707. Springer, Cham. https://doi.org/10.1007/978-3-031-61044-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61044-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61043-1

  • Online ISBN: 978-3-031-61044-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics