Luana Porciuncula
Fabio Q B Da Silva
ABSTRACT
Research into Fish Tank Virtual Reality (FTVR) techniques commonly uses specific wearable sensors such as infrared cameras, glasses, or helmets to estimate the position of the user’s eyes. However, using only existing RGB cameras on laptops is more affordable for users as they are already widely available on devices. In this work, we continue and increase previous results using face feature detection software to implement the FTVR technique in 3D applications using standard laptops without needing extra devices. The main contribution is to enhanced the Parallax Engine solution to: 1)This work also defines and tests a protocol to evaluate the user experience and the sense of the presence of the Parallax Engine; 2) adding a new dimension in parallax mode (Parallax3DoF), with freedom of horizontal, vertical and depth movement with cameras in 3D scenes; 3) we eliminate the need for Python servers and the intermediation of network communication between Unity and these servers; and 4) improve integration, performance, and user experience of monoscopic FTVR mode (FishTank), with the virtual camera locked in the 3D environment of laptop screens. Parallax Engine uses Google Media Pipe for face mesh detection via an open-source implementation called the Media Pipe Plugin, available for Unity. We evaluated the proposed solution and almost doubled the FPS of the tool. In addition, we did not find fatigue problems, the interface was considered very good on the SUS scale, and the immersion was considered acceptable. We also compared the immersion and user experience of the FishTank and Parallax3DoF visualization modes, and the users evaluated better the Parallax3DoF in both. We aim in future work to explore this technique further and test it with more users.
- Ritu Agarwal and Elena Karahanna. 2000. Time flies when you’re having fun: Cognitive absorption and beliefs about information technology usage. MIS quarterly (2000), 665–694.Google Scholar
Digital Library
- Thomas Alsop. 2022. Global notebook personal computer (PC) shipments from 2010 to 2026. https://www.statista.com/statistics/269048/worldwide-portable-pc-shipment-forecast/.Google Scholar
- Christof Baron. 2022. Price per unit in the computer hardware market in Europe from 2013 to 2026, by segment. https://www.statista.com/forecasts/1256765/prices-per-unit-computer-hardware-europe.Google Scholar
- Patrick Bourdot, Martin Dromigny, and Laurent Arnal. 1999. Virtual navigation fully controlled by head tracking. In Proc. of International Scientific Workshop on Virtual Reality and Prototyping.Google Scholar
- John Brooke 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4–7.Google Scholar
- Emily Brown and Paul Cairns. 2004. A grounded investigation of game immersion. In CHI’04 extended abstracts on Human factors in computing systems. 1297–1300.Google Scholar
- Weiya Chen, Anthony Plancoulaine, Nicolas Férey, Damien Touraine, Julien Nelson, and Patrick Bourdot. 2013. 6DoF Navigation in Virtual Worlds: Comparison of Joystick-Based and Head-Controlled Paradigms. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology (Singapore) (VRST ’13). Association for Computing Machinery, New York, NY, USA, 111–114. https://doi.org/10.1145/2503713.2503754Google ScholarDigital Library
- FMI. 2022. Head Mounted Display Market - About the Report. https://www.futuremarketinsights.com/reports/global-head-mounted-display-market#.Google Scholar
- Juliano Franz, Anderson Maciel, and Luciana Nedel. 2013. Assessment of a User Centered Interface for Teleoperation and 3D Environments. In Proceedings of the 28th Annual ACM Symposium on Applied Computing (Coimbra, Portugal) (SAC ’13). Association for Computing Machinery, New York, NY, USA, 953–960. https://doi.org/10.1145/2480362.2480546Google ScholarDigital Library
- Teresa Hirzle, Jan Gugenheimer, Florian Geiselhart, Andreas Bulling, and Enrico Rukzio. 2019. A design space for gaze interaction on head-mounted displays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Jarbas Jácome, Arlindo Gomes, Willams de Lima Costa, Lucas Silva Figueiredo, Jader Abreu, Luana Porciuncula, Pedro K Brant, Luis EM Alves, Walter FM Correia, Veronica Teichrieb, 2021. Parallax Engine: Head Controlled Motion Parallax Using Notebooks’ RGB Camera. In Symposium on Virtual and Augmented Reality. 137–146.Google Scholar
- Charlene Jennett, Anna L Cox, Paul Cairns, Samira Dhoparee, Andrew Epps, Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of immersion in games. International journal of human-computer studies 66, 9 (2008), 641–661.Google ScholarDigital Library
- Robert S Kennedy, Norman E Lane, Kevin S Berbaum, and Michael G Lilienthal. 1993. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 3 (1993), 203–220.Google Scholar
- Davis E King. 2009. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research 10 (2009), 1755–1758.Google ScholarDigital Library
- Sirisilp Kongsilp and Matthew N. Dailey. 2017. Motion parallax from head movement enhances stereoscopic displays by improving presence and decreasing visual fatigue. Displays 49 (2017), 72–79. https://doi.org/10.1016/j.displa.2017.07.001Google ScholarCross Ref
- Sirisilp Kongsilp and Matthew N. Dailey. 2018. User Behavior and the Importance of Stereo for Depth Perception in Fish Tank Virtual Reality. Presence: Teleoperators and Virtual Environments 27, 2 (02 2018), 206–225. https://doi.org/10.1162/pres_a_00327Google ScholarCross Ref
- Jason Lawrence, Dan B Goldman, Supreeth Achar, Gregory Major Blascovich, Joseph G. Desloge, Tommy Fortes, Eric M. Gomez, Sascha Häberling, Hugues Hoppe, Andy Huibers, Claude Knaus, Brian Kuschak, Ricardo Martin-Brualla, Harris Nover, Andrew Ian Russell, Steven M. Seitz, and Kevin Tong. 2021. Project Starline: A high-fidelity telepresence system. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 40(6) (2021).Google ScholarDigital Library
- Raphael Leroy. 2021. Immersion, Flow and Usability in Video Games. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA ’21). Association for Computing Machinery, New York, NY, USA, Article 483, 7 pages. https://doi.org/10.1145/3411763.3451514Google ScholarDigital Library
- Edmund F. LoPresti, David M. Brienza, and Jennifer Angelo. 2002. Head-operated computer controls: effect of control method on performance for subjects with and without disability. Interacting with Computers 14, 4 (07 2002), 359–377. https://doi.org/10.1016/S0953-5438(01)00058-3Google ScholarCross Ref
- Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. MediaPipe: A Framework for Building Perception Pipelines. https://doi.org/10.48550/ARXIV.1906.08172Google ScholarCross Ref
- Michael C Medlock, Dennis Wixon, Mick McGee, and Dan Welsh. 2005. The rapid iterative test and evaluation method: Better products in less time. In Cost-justifying usability. Elsevier, 489–517.Google Scholar
- Dionysios Georgios Papadimitriou. 2019. User Experience in Virtual Reality, conducting an evaluation on multiple characteristics of a Virtual Reality Experience. Master’s thesis.Google Scholar
- George Papandreou, Tyler Zhu, Liang-Chieh Chen, Spyros Gidaris, Jonathan Tompson, and Kevin Murphy. 2018. PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model. https://doi.org/10.48550/ARXIV.1803.08225Google ScholarCross Ref
- Eduardo Rodrigues, Lucas Silva Figueiredo, Lucas Maggi, Edvar Neto, Layon Tavares Bezerra, João Marcelo Teixeira, and Veronica Teichrieb. 2017. Mixed Reality TVs: Applying Motion Parallax for Enhanced Viewing and Control Experiences on Consumer TVs. In 2017 19th Symposium on Virtual and Augmented Reality (SVR). 319–330. https://doi.org/10.1109/SVR.2017.48Google ScholarCross Ref
- Pejman Saeghe, Gavin Abercrombie, Bruce Weir, Sarah Clinch, Stephen Pettifer, and Robert Stevens. 2020. Augmented Reality and Television: Dimensions and Themes. In ACM International Conference on Interactive Media Experiences (Cornella, Barcelona, Spain) (IMX ’20). Association for Computing Machinery, New York, NY, USA, 13–23. https://doi.org/10.1145/3391614.3393649Google ScholarDigital Library
- Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.Google ScholarDigital Library
Index Terms
- Mixed Reality Laptops: A Protocol to Evaluate Enhanced Viewing Experiences on Consumer Laptops Using Parallax Engine
Recommendations
Parallax Engine: Head Controlled Motion Parallax Using Notebooks’ RGB Camera
SVR '21: Proceedings of the 23rd Symposium on Virtual and Augmented RealityResearch on the Fish Tank Virtual Reality (FTVR) technique commonly uses specific sensors (e.g. infrared cameras and LEDs on glasses) to estimate user’s eye position. However, estimating the face position with an RGB camera is becoming more accessible. ...
The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception
APGV '08: Proceedings of the 5th symposium on Applied perception in graphics and visualizationAs the use of virtual and augmented reality applications becomes more common, the need to fully understand how observers perceive spatial relationships grows more critical. One of the key requirements in engineering a practical virtual or augmented ...
A Motion Parallax Rendering Approach to Real-time Stereoscopic Visualization for Aircraft Virtual Assembly
ICVRV '12: Proceedings of the 2012 International Conference on Virtual Reality and VisualizationAs a cue to depth perception, motion parallax can improve stereoscopic visualization to a level more like human natural vision. And stereoscopic visualization with motion parallax rendering can lessen the fatigue when people are long-time immersed in ...
Comments