Konstantinos Tsigkounis
Andreas Komninos
ABSTRACT
In this paper, we present a novel system aimed at replacing large-display information dashboards in industrial control rooms, with a flexible, dynamic and reconfigurable immersive virtual reality environment, which can afford high mobility without constraints, to remote engineers. In this context, we investigate the role of semantic and spatial cues for delivering event notifications within the control environment, and present empirical evidence from a controlled laboratory study, simulating a marine industrial environment. We find that spatial and semantic cues can both offer significant benefits to operator awareness and their combination can significantly improve the findability and response time to particular information in the extended information space surrounding the user.
- Shamel Addas and Alain Pinsonneault. 2015. The Many Faces of Information Technology Interruptions: A Taxonomy and Preliminary Investigation of Their Performance Effects. Information Systems Journal 25, 3 (2015), 231–273. https://doi.org/10.1111/isj.12064 Google Scholar
Digital Library
- SatriadiKadek Ananta, EnsBarrett, CordeilMaxime, CzaudernaTobias, and JennyBernhard. 2020. Maps Around Me. Proceedings of the ACM on Human-Computer Interaction (Nov. 2020). https://doi.org/10.1145/3427329Google Scholar
- Christopher Andrews, Alex Endert, and Chris North. 2010. Space to Think: Large High-Resolution Displays for Sensemaking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 55–64. https://doi.org/10.1145/1753326.1753336 Google ScholarDigital Library
- Joakim D. Bergroth, Hanna M. K. Koskinen, and Jari O. Laarni. 2018. Use of Immersive 3-D Virtual Reality Environments in Control Room Validations. Nuclear Technology 202, 2-3 (June 2018), 278–289. https://doi.org/10.1080/00295450.2017.1420335Google ScholarCross Ref
- Manousos Bouloukakis, Nikolaos Partarakis, Ioannis Drossis, Manos Kalaitzakis, and Constantine Stephanidis. 2019. Virtual Reality for Smart City Visualization and Monitoring. In Mediterranean Cities and Island Communities: Smart, Sustainable, Inclusive and Resilient, Anastasia Stratigea and Dimitris Kavroudakis (Eds.). Springer International Publishing, Cham, 1–18. https://doi.org/10.1007/978-3-319-99444-4_1Google Scholar
- Steven Vanden Broucke and Nikos Deligiannis. 2019. Visualization of Real-Time Heterogeneous Smart City Data Using Virtual Reality. In 2019 IEEE International Smart Cities Conference (ISC2). 685–690. https://doi.org/10.1109/ISC246665.2019.9071699Google Scholar
- Marco Cavallo, Mishal Dholakia, Matous Havlena, Kenneth Ocheltree, and Mark Podlaseck. 2019. Dataspace: A Reconfigurable Hybrid Reality Environment for Collaborative Information Analysis. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 145–153. https://doi.org/10.1109/VR.2019.8797733Google Scholar
- Eunhee Chang, Hyun Taek Kim, and Byounghyun Yoo. 2020. Virtual Reality Sickness: A Review of Causes and Measurements. International Journal of Human–Computer Interaction 36, 17 (Oct. 2020), 1658–1682. https://doi.org/10.1080/10447318.2020.1778351Google ScholarCross Ref
- Peter Cheng, Sharon Wood, and Richard Cox. 2007. Dimensions of Attentional Processing. In Proceedings of the UBIComp Workshop on Attention Management in Ubiquitous Computing Environments. Innsbruck, Austria.Google Scholar
- Lorenzo Damiani, Melissa Demartini, Guido Guizzi, Roberto Revetria, and Flavio Tonelli. 2018. Augmented and Virtual Reality Applications in Industrial Systems: A Qualitative Review towards the Industry 4.0 Era. IFAC-PapersOnLine 51, 11 (Jan. 2018), 624–630. https://doi.org/10.1016/j.ifacol.2018.08.388Google ScholarCross Ref
- Ícaro Aragão Fonseca and Henrique Murilo Gaspar. 2021. Challenges When Creating a Cohesive Digital Twin Ship: A Data Modelling Perspective. Ship Technology Research 68, 2 (May 2021), 70–83. https://doi.org/10.1080/09377255.2020.1815140Google ScholarCross Ref
- Emanuele Frontoni, Jelena Loncarski, Roberto Pierdicca, Michele Bernardini, and Michele Sasso. 2018. Cyber Physical Systems for Industry 4.0: Towards Real Time Virtual Reality in Smart Manufacturing. In Augmented Reality, Virtual Reality, and Computer Graphics(Lecture Notes in Computer Science), Lucio Tommaso De Paolis and Patrick Bourdot (Eds.). Springer International Publishing, Cham, 422–434. https://doi.org/10.1007/978-3-319-95282-6_31Google Scholar
- Norberto Martínez García, Mariano Azzur Hernández Contreras, Mariana Silva-Ortega, Verónica García Valenzuela, and Teresita de Jesús Olivares. 2020. Creation of Virtual Environment for Teaching Mechanical Maintenance on a Tug Ship. In 2020 IEEE International Conference on Engineering Veracruz (ICEV). 1–6. https://doi.org/10.1109/ICEV50249.2020.9289662Google ScholarCross Ref
- Leandro Barbosa S. Gatto, Antônio Carlos A. Mól, Isaac J. A. Luquetti dos Santos, Carlos Alexandre F. Jorge, and Ana Paula Legey. 2013. Virtual Simulation of a Nuclear Power Plant’s Control Room as a Tool for Ergonomic Evaluation. Progress in Nuclear Energy 64 (April 2013), 8–15. https://doi.org/10.1016/j.pnucene.2012.11.006Google Scholar
- Sarthak Ghosh, Lauren Winston, Nishant Panchal, Philippe Kimura-Thollander, Jeff Hotnog, Douglas Cheong, Gabriel Reyes, and Gregory D. Abowd. 2018. NotifiVR: Exploring Interruptions and Notifications in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics 24, 4 (April 2018), 1447–1456. https://doi.org/10.1109/TVCG.2018.2793698 Google ScholarDigital Library
- Ioannis Giannopoulos, Andreas Komninos, and John Garofalakis. 2017. Natural Interaction with Large Map Interfaces in VR. In Proceedings of the 21st Pan-Hellenic Conference on Informatics(PCI 2017). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3139367.3139424 Google ScholarDigital Library
- Ching-Yu Hsieh, Yi-Shyuan Chiang, Hung-Yu Chiu, and Yung-Ju Chang. 2020. Bridging the Virtual and Real Worlds: A Preliminary Study of Messaging Notifications in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376228 Google ScholarDigital Library
- Miao Hu, Xianzhuo Luo, Jiawen Chen, Young Choon Lee, Yipeng Zhou, and Di Wu. 2021. Virtual Reality: A Survey of Enabling Technologies and Its Applications in IoT. Journal of Network and Computer Applications 178 (March 2021), 102970. https://doi.org/10.1016/j.jnca.2020.102970Google ScholarCross Ref
- Shamsi T. Iqbal and Eric Horvitz. 2007. Disruption and Recovery of Computing Tasks: Field Study, Analysis, and Directions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 677–686. https://doi.org/10.1145/1240624.1240730 Google ScholarDigital Library
- David Jones, Chris Snider, Aydin Nassehi, Jason Yon, and Ben Hicks. 2020. Characterising the Digital Twin: A Systematic Literature Review. CIRP Journal of Manufacturing Science and Technology 29 (May 2020), 36–52. https://doi.org/10.1016/j.cirpj.2020.02.002Google Scholar
- Vasiliki Liagkou, Dimitrios Salmas, and Chrysostomos Stylios. 2019. Realizing Virtual Reality Learning Environment for Industry 4.0. Procedia CIRP 79 (Jan. 2019), 712–717. https://doi.org/10.1016/j.procir.2019.02.025Google Scholar
- Christian Linn, Simon Bender, Joshua Prosser, Kevin Schmitt, and Dirk Werth. 2017. Virtual Remote Inspection — A New Concept for Virtual Reality Enhanced Real-Time Maintenance. In 2017 23rd International Conference on Virtual System Multimedia (VSMM). 1–6. https://doi.org/10.1109/VSMM.2017.8346304Google ScholarCross Ref
- Lee Lisle. 2020. [DC]The Immersive Space to Think: Immersive Analytics for Multimedia Data. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 551–552. https://doi.org/10.1109/VRW50115.2020.00127Google ScholarCross Ref
- Mark Mcgill, Aidan Kehoe, Euan Freeman, and Stephen Brewster. 2020. Expanding the Bounds of Seated Virtual Workspaces. ACM Transactions on Computer-Human Interaction 27, 3 (May 2020), 13:1–13:40. https://doi.org/10.1145/3380959 Google ScholarDigital Library
- Leonardo Pavanatto, Chris North, Doug A. Bowman, Carmen Badea, and Richard Stoakley. 2021. Do We Still Need Physical Monitors? An Evaluation of the Usability of AR Virtual Monitors for Productivity Work. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). 759–767. https://doi.org/10.1109/VR50410.2021.00103Google Scholar
- Patrick Reipschlager, Tamara Flemisch, and Raimund Dachselt. 2021. Personal Augmented Reality for Information Visualization on Large Interactive Displays. IEEE Transactions on Visualization and Computer Graphics 27, 2 (Feb. 2021), 1182–1192. https://doi.org/10.1109/TVCG.2020.3030460Google ScholarCross Ref
- Vasja Roblek, Maja Meško, and Alojz Krapež. 2016. A Complex View of Industry 4.0. SAGE Open 6, 2 (April 2016), 2158244016653987. https://doi.org/10.1177/2158244016653987Google ScholarCross Ref
- Juan Jesús Roldán, Elena Crespo, Andrés Martín-Barrio, Elena Peña-Tapia, and Antonio Barrientos. 2019. A Training System for Industry 4.0 Operators in Complex Assemblies Based on Virtual Reality and Process Mining. Robotics and Computer-Integrated Manufacturing 59 (Oct. 2019), 305–316. https://doi.org/10.1016/j.rcim.2019.05.004Google Scholar
- Rufat Rzayev, Sven Mayer, Christian Krauter, and Niels Henze. 2019. Notification in VR: The Effect of Notification Placement, Task and Environment. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’19). Association for Computing Machinery, New York, NY, USA, 199–211. https://doi.org/10.1145/3311350.3347190 Google ScholarDigital Library
- Muhammad Saleem, Hugo E. Valle, Stephen Brown, Veronica I. Winters, and Akhtar Mahmood. 2018. The Hiperwall Tiled-Display Wall System for Big-Data Research. Journal of Big Data 5, 1 (Oct. 2018), 41. https://doi.org/10.1186/s40537-018-0150-7Google ScholarCross Ref
- Alper Sarikaya, Michael Correll, Lyn Bartram, Melanie Tory, and Danyel Fisher. 2019. What Do We Talk About When We Talk About Dashboards?IEEE Transactions on Visualization and Computer Graphics 25, 1 (Jan. 2019), 682–692. https://doi.org/10.1109/TVCG.2018.2864903Google Scholar
- Ahm Shamsuzzoha, Rayko Toshev, Viet Vu Tuan, Timo Kankaanpaa, and Petri Helo. 2019. Digital Factory – Virtual Reality Environments for Industrial Training and Maintenance. Interactive Learning Environments 0, 0 (June 2019), 1–24. https://doi.org/10.1080/10494820.2019.1628072Google Scholar
- Elaine C. Silva, Carla A. G. Sirqueira, and Fernanda G. S. Pinto. 2021. The Immersion in Virtual Reality of Control Room Activity. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021)(Lecture Notes in Networks and Systems), Nancy L. Black, W. Patrick Neumann, and Ian Noy(Eds.). Springer International Publishing, Cham, 82–88. https://doi.org/10.1007/978-3-030-74608-7_11Google Scholar
- Daniel J Simons and Christopher F Chabris. 1999. Gorillas in Our Midst: Sustained Inattentional Blindness for Dynamic Events. Perception 28, 9 (Sept. 1999), 1059–1074. https://doi.org/10.1068/p281059Google Scholar
- Cheri Speier, Iris Vessey, and Joseph S. Valacich. 2003. The Effects of Interruptions, Task Complexity, and Information Presentation on Computer-Supported Decision-Making Performance. Decision Sciences 34, 4 (2003), 771–797. https://doi.org/10.1111/j.1540-5414.2003.02292.xGoogle ScholarCross Ref
- Sławomir Konrad Tadeja, Yupu Lu, Pranay Seshadri, and Per Ola Kristensson. 2020. Digital Twin Assessments in Virtual Reality: An Explorational Study with Aeroengines. In 2020 IEEE Aerospace Conference. 1–13. https://doi.org/10.1109/AERO47225.2020.9172389Google Scholar
- Matt Whitlock, Stephen Smart, and Danielle Albers Szafir. 2020. Graphical Perception for Immersive Analytics. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 616–625. https://doi.org/10.1109/VR46266.2020.00084Google Scholar
- Yin Yang, Chen Shuai, and Gui Fan. 2019. The Technological Development and Prospect of Unmanned Surface Vessel. In 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). 717–722. https://doi.org/10.1109/IMCEC46724.2019.8983889Google Scholar
- Mohsen Zare, Maxime Larique, Sébastien Chevriau, and Jean-Claude Sagot. 2019. Application of Virtual Reality to Improve Physical Ergonomics in a Control Room of a Chemical Industry. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018)(Advances in Intelligent Systems and Computing), Sebastiano Bagnara, Riccardo Tartaglia, Sara Albolino, Thomas Alexander, and Yushi Fujita(Eds.). Springer International Publishing, Cham, 296–308. https://doi.org/10.1007/978-3-319-96077-7_31Google ScholarCross Ref
- André Zenner, Marco Speicher, Sören Klingner, Donald Degraen, Florian Daiber, and Antonio Krüger. 2018. Immersive Notification Framework: Adaptive & Plausible Notifications in Virtual Reality. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188505 Google ScholarDigital Library
Index Terms
- Monitoring Maritime Industry 4.0 Systems through VR Environments
Index terms have been assigned to the content through auto-classification.
Recommendations
Spatial Multimodal Alert Cues for Virtual Reality Control Environments in Industry 4.0
CHIGREECE '23: Proceedings of the 2nd International Conference of the ACM Greek SIGCHI ChapterThe upcoming 4th Industrial Revolution (Industry 4.0) is based on data collection and analysis from mechanical equipment equipped with sensors. Typically, the operating parameters of an installation are controlled by engineers who are brought into ...
Extended Reality (XR) Toward Building Immersive Solutions: The Key to Unlocking Industry 4.0
When developing XR applications for Industry 4.0, it is important to consider the integration of visual displays, hardware components, and multimodal interaction techniques that are compatible with the entire system. The potential use of multimodal ...
Notification in VR: The Effect of Notification Placement, Task and Environment
CHI PLAY '19: Proceedings of the Annual Symposium on Computer-Human Interaction in PlayVirtual reality (VR) is commonly used for entertainment applications but is also increasingly employed for a large number of use cases such as digital prototyping or training workers. Here, VR is key to present an immersive secondary world. VR enables ...
Comments