Abstract
We present a framework for alternative reality (XR) technologies to enable an understanding of what constitutes an XR environment when used in the context of design and engineering of large, complex systems. The framework provides guidelines for implementing the corresponding desired human sensory experience. This work is founded on the existing literature which defines theoretical Spectra, such as Fidelity, to systematically characterize an XR environment taxonomy. We identify landmarks for four XR categories within these Spectra and provide definitions that can used to establish a common vernacular. We further map these to specific human sensing modalities that are influenced by XR, such as tactility and vision, and define the technical requirements needed to augment the human experience in the desired XR environment. Finally, we connect the theoretical elements to the technical requirements to create an integrated XR framework. The utility of this framework is demonstrated in a case study addressing the use of XR technologies for five stakeholder groups involved in the evaluation of spacecraft habitat design and operations. This demonstrates the utility of the proposed XR taxonomy in a spacecraft habitat design process, which could be extended to other similar applications.
This is a preview of subscription content, log in via an institution to check access.
adapted from Milgram and Kishino. Increasing virtual abstraction is achieved from left to right
Similar content being viewed by others
References
Acemyan CZ, Kortum P (2018) Does the type of presentation medium impact assessments of the built environment? An examination of environmental usability ratings across three modes of presentation. J Environ Psychol 56(April):30–35. https://doi.org/10.1016/j.jenvp.2018.02.006
Adams E (2017) Ford designers use microsoft hololens augmented reality to make better cars|WIRED. September 2017. https://www.wired.com/story/ford-design-microsoft-hololens/.
Arayici Y, Coates P, Koskela L, Kagioglou M, Usher C, K. O’reilly. (2011) BIM adoption and implementation for architectural practices. Struct Surv 29(1):7–25
Azuma RT (1997) A survey of augmented reality. Pres Teleoper Virt Environ 6(4):355–385
Begault, D. R. T. (2000). 3-D Sound for virtual reality and multimedia. https://ntrs.nasa.gov/search.jsp?R=20010044352.
Bowman DA, Ryan PM (2007) Virtual reality: How much immersion is enough? Computer 40(7):36–43. https://doi.org/10.1109/MC.2007.257
Castronovo F, Dragana N, Liu Y, Messner J (2013) An evaluation of immersive virtual reality systems for design reviews. Comput Aided Design Appl 9:892
Cohen, M. (2012a). Mockups 101: Code and standard research for space habitat analogues. In: American Institute of Aeronautics and Astronautics. https://doi.org/10.2514/6.2012-5153.
Cohen, M. (2012b). Mockups 101: Code and standard research for space habitat analogues. In: American Institute of Aeronautics and Astronautics. https://doi.org/10.2514/6.2012-5153.
Dubois E, Philip G, Laurence N (eds) (2010) The engineering of mixed reality systems. Human–computer interaction series. Springer, London
Eastman CM (ed) (2008) BIM handbook: a guide to building information modeling for owners, managers, designers, engineers, and contractors. Wiley, Hoboken
Engelberg D, Seffah A (2002) A Framework for rapid mid-fidelity prototyping of web sites. In: Hammond J, Gross T, Wesson J (eds) Usabilit, vol 99. Springer, Boston, pp 203–215. https://doi.org/10.1007/978-0-387-87035610-5_14
Fishkin KP (2004) A taxonomy for and analysis of tangible interfaces. Pers Ubiquit Comput 8(5):347. https://doi.org/10.1007/s00779-004-0297-4
Goulding J, Rahimian F, Wang X (2014) Virtual reality-based cloud BIM platform for integrated AEC projects. J Inf Technol Constr 19:308–325
Gugenheimer J, Stemasov E, Frommel J, Rukzio E (2017). ShareVR: enabling co-located experiences for virtual reality between HMD and non-HMD users. In: ACM Press, pp 4021–433. https://doi.org/10.1145/3025453.3025683.
Hale KS, Stanney KM, Malone L (2009) Enhancing virtual environment spatial awareness training and transfer through tactile and vestibular cues. Ergonomics 52(2):187–203. https://doi.org/10.1080/00140130802376000
Hammond W (1999) Space transportation: A systems approach to analysis and design AIAA educational series. American Institute of Aeronautics and Astronautics, Reston
Hays T (1980) Simulation systems technical area. Technical Report 490. U.S. Army Research Institute of Behavioral and Social Sciences.
Hettinger L, Riccio G (1992) Visually induced motion sickness in virtual environments. Pres Teleoper Virt Environ 1(3):306
Heydarian A, Carneiro JP, Gerber D, Becerik-Gerber B, Hayes T, Wood W (2015) Immersive virtual environments versus physical built environments: a benchmarking study for building design and user-built environment explorations. Autom Constr 54(June):116–126. https://doi.org/10.1016/j.autcon.2015.03.020
Higdon KP, Klaus DM (2008) Effective integration of rapid prototyping into multidisciplinary design optimization for the development of human spacecraft. In: American Society of Civil Engineers, pp 1–10. https://doi.org/10.1061/40988(323)96.
Ishii H (2008) Tangible bits: beyond pixels. In: xv. ACM Press. https://doi.org/10.1145/1347390.1347392.
Kanas N, Manzey D (2008) Space psychology and psychiatry. Space technology library, 2nd edn. Springer, El Segundo
Kelly DH (1979) Motion and vision. II. Stabilized spatio-temporal threshold surface. J Opt Soc Am 69(10):10
Kim M, Jeon C, Kim J (2017) A study on immersion and presence of a portable hand haptic system for immersive virtual reality. Sensors; Basel 17(5):1141
Klaus DM, Higdon KP (2009) Academic principles of human space habitat design. In: International conference on environmental systems, vol 8
Larsson P, Väljamäe A, Västfjäll D, Tajadura-Jiménez A, Kleiner M (2010) Auditory-induced presence in mixed reality environments and related technology. In: Dubois E, Gray P, Nigay L (eds) The engineering of mixed reality systems. Springer, London, pp 143–163. https://doi.org/10.1007/978-1-84882-733-2_8
Leithinger D, Follmer S, Olwal A, Ishii H (2015) Shape displays: spatial interaction with dynamic physical form. IEEE Comput Graph Appl 35(5):5–11. https://doi.org/10.1109/MCG.2015.111
Lentz T, Schröder D, Vorländer M, Assenmacher I (2007) Virtual reality system with integrated sound field simulation and reproduction. EURASIP J Adv Signal Process 2007:1. https://doi.org/10.1155/2007/70540
Liu D, Macchiarella ND, Vincenzi DA (2009) Simulation fidelity. Hum Factors Simul Training 13:561
Mackay, W. (1998). Augmented reality: Linking real and virtual worlds a new paradigm for interacting with computers. In: ACM conference on advanced visual interfaces Proceedings of AVI’9. ACM Press, pp 13–21.
Maldovan KD, Messner JI, Faddoul M (2006) Framework for reviewing mockups in an immersive environment, vol 7. https://www.researchgate.net/profile/John_Messner2/publication/228399774_Framework_for_Reviewing_Mockups_in_an_Immersive_Environment/links/004635231e625e739f000000.pdf
Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst 77(12):1321–1329
Mugge W, David AA, Alfred CS, van der Frans CT, Helm JHA, Carel GMM (2013) Force control in the absence of visual and tactile feedback. Exp Brain Res 224(4):635–645. https://doi.org/10.1007/s00221-012-3341-z
Mulenburg GM, Gundo DP (2002) Design by Prototype: examples from the National Aeronautics and Space Administration. Miami, FL.
NASA (1995) NASA systems engineering handbook. NASA.
NASA (2011) NASA space flight human system standard volume 2: Human factors, habitability, and environmental health. NASA-STD-3001 VOL II.
Nolle S, Gudrun K (2006) Augmented reality as a comparison tool in automotive industry. In: IEEE, pp 249–250. https://doi.org/10.1109/ISMAR.2006.297829.
Paige JB, Morin KH (2013) Simulation fidelity and cueing: a systematic review of the literature. Clin Simul Nurs 9(11):e481–e489. https://doi.org/10.1016/j.ecns.2013.01.001
Pamungkas DS, Ward K (2016) Electro-tactile feedback system to enhance virtual reality experience. Int J Comput Theory Eng 8(6):465–470. https://doi.org/10.7763/IJCTE.2016.V8.1090
Picinali L, Amandine A, Michel D, Brian FGK (2014) Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge. Int J Hum Comput Stud 72(4):393–407. https://doi.org/10.1016/j.ijhcs.2013.12.008
Pontonnier C, Dumont G, Samani A, Madeleine P, Badawi M (2014) Designing and evaluating a workstation in real and virtual environment: toward virtual reality based ergonomic design sessions. J Multimodal User Interfaces 8(2):199–208. https://doi.org/10.1007/s12193-013-0138-8
Riecke BE, Jörg S-P, Franck C, Heinrich H B (n.d.) Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality, vol 9. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.490.7866&rep=rep1&type=pdf
Robinett W (1992) Synthetic experience: a proposed taxonomy. Pres Teleoper Virt Environ 1(2):229–247
Rudd J, Stern K, Isensee S (1996) Low vs. high-fidelity prototyping debate. Interactions 3:76–85
Ryu J, Naoki H, Makoto S, Masashi S, Ryuzo O (2007) Application of human-scale immersive VR system for environmental design assessment: a proposal for an architectural design evaluation tool. J Asian Archit Build Eng 6(1):57–64. https://doi.org/10.3130/jaabe.6.57
Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virt Real 15(1):5–20. https://doi.org/10.1007/s10055-009-0153-y
Sheridan TB (1992) Musings on telepresence and virtual presence. Pres Teleoper Virt Environ 1(1):120–126
Västfjäll D (2003) The subjective sense of presence, emotion recognition, and experienced emotions in auditory virtual environments. Cyber Psychol Behav 6(2):181–188. https://doi.org/10.1089/109493103321640374
Whitmore M, McGuire K, Margerum S, Thompson S, Allen C, Bowen C, Adelstein B, Schuh S, Byrne V, Wong D (2013) Evidence report: risk of incompatible vehicle/habitat design. NASA Human Research Program, Johnson Space Center, Houston
Zeltzer D (1992) Autonomy, interaction, and presence. Pres Teleoper Virt Environ 1(1):127–132
Zimmermann P (2008) Virtual reality aided design: A survey of the use of VR in automotive industry. In: Talaba D, Amditis A (eds) Product engineering. Springer, Dordrecht, pp 277–296
Acknowledgements
This work was funded by the NASA Human Research Program, Grant 80NSSC18K0198.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors have no conflict of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix 1: interview protocol
Appendix 1: interview protocol
Rights and permissions
About this article
Cite this article
Anderson, A., Boppana, A., Wall, R. et al. Framework for developing alternative reality environments to engineer large, complex systems. Virtual Reality 25, 147–163 (2021). https://doi.org/10.1007/s10055-020-00448-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-020-00448-4