skip to main content
research-article

Exploring Redirection and Shifting Techniques to Mask Hand Movements from Shoulder-Surfing Attacks during PIN Authentication in Virtual Reality

Published: 24 September 2024 Publication History

Abstract

The proliferation of mobile Virtual Reality (VR) headsets shifts our interaction with virtual worlds beyond our living rooms into shared spaces. Consequently, we are entrusting more and more personal data to these devices, calling for strong security measures and authentication. However, the standard authentication method of such devices - entering PINs via virtual keyboards - is vulnerable to shoulder-surfing, as movements to enter keys can be monitored by an unnoticed observer. To address this, we evaluated masking techniques to obscure VR users' input during PIN authentication by diverting their hand movements. Through two experimental studies, we demonstrate that these methods increase users' security against shoulder-surfing attacks from observers without excessively impacting their experience and performance. With these discoveries, we aim to enhance the security of future VR authentication without disrupting the virtual experience or necessitating additional hardware or training of users.

Supplemental Material

MP4 File
Supplemental video
ZIP File
This auxiliary material contains the subtitles (SRT-Format) for the video figure.

References

[1]
Yomna Abdelrahman, Florian Mathis, Pascal Knierim, Axel Kettler, Florian Alt, and Mohamed Khamis. 2022. CueVR: Studying the Usability of Cue-Based Authentication for Virtual Reality. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces (Frascati, Rome, Italy) (AVI 2022). Association for Computing Machinery, New York, NY, USA, Article 34, 9 pages. https://doi.org/10.1145/3531073.3531092
[2]
Parastoo Abtahi and Sean Follmer. 2018. Visuo-Haptic Illusions for Improving the Perceived Performance of Shape Displays. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3173724
[3]
Karan Ahuja, Rahul Islam, Varun Parashar, Kuntal Dey, Chris Harrison, and Mayank Goel. 2018. EyeSpyVR: Interactive Eye Sensing Using Off-the-Shelf, Smartphone-Based VR Headsets. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 2, 2, Article 57 (jul 2018), 10 pages. https://doi.org/10.1145/3214260
[4]
Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 1968--1979. https://doi.org/10.1145/2858036.2858226
[5]
Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2014. Displaying Shapes with Various Types of Surfaces Using Visuo-Haptic Interaction. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (Edinburgh, Scotland) (VRST '14). Association for Computing Machinery, New York, NY, USA, 191--196. https://doi.org/10.1145/2671015.2671028
[6]
Raoul Bickmann, Celine Tran, Ninja Ruesch, and Katrin Wolf. 2019. Haptic Illusion Glove: A Glove for Illusionary Touch Feedback When Grasping Virtual Objects. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany) (MuC'19). Association for Computing Machinery, New York, NY, USA, 565--569. https://doi.org/10.1145/3340764.3344459
[7]
Matthew Botvinick and Jonathan Cohen. 1998. Rubber hands `feel' touch that eyes see. Nature, Vol. 391, 6669 (01 Feb 1998), 756--756. https://doi.org/10.1038/35784
[8]
Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 3718--3728. https://doi.org/10.1145/3025453.3025753
[9]
Martin Feick, Kora P. Regitz, Anthony Tang, Tobias Jungbluth, Maurice Rekrut, and Antonio Krüger. 2023. Investigating Noticeable Hand Redirection in Virtual Reality using Physiological and Interaction Data. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, New York, NY, USA, 194--204. https://doi.org/10.1109/VR55154.2023.00035
[10]
Ceenu George, Mohamed Khamis, Daniel Buschek, and Heinrich Hussmann. 2019. Investigating the Third Dimension for Authentication in Immersive Virtual Reality and in the Real World. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, New York, NY, USA, 277--285. https://doi.org/10.1109/VR.2019.8797862
[11]
Ceenu George, Mohamed Khamis, Emanuel Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017. Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality. In Network and Distributed System Security Symposium (NDSS 2017). https://doi.org/10.14722/usec.2017.23028
[12]
Benoit Geslain, Simon Besga, Flavien Lebrun, and Gilles Bailly. 2022. Generalizing the Hand Redirection Function in Virtual Reality. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces (Frascati, Rome, Italy) (AVI 2022). Association for Computing Machinery, New York, NY, USA, Article 33, 9 pages. https://doi.org/10.1145/3531073.3531100
[13]
Eric J. Gonzalez, Parastoo Abtahi, and Sean Follmer. 2020. REACH: Extending the Reachability of Encountered-type Haptics Devices through Dynamic Redirection in VR. Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (2020).
[14]
Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50, 9 (Oct. 2006), 904--908. https://doi.org/10.1177/154193120605000909
[15]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139--183. https://doi.org/10.1016/S0166--4115(08)62386--9
[16]
Mohamed Khamis, Carl Oechsner, Florian Alt, and Andreas Bulling. 2018. VRpursuits: Interaction in Virtual Reality Using Smooth Pursuit Eye Movements. In Proceedings of the 2018 International Conference on Advanced Visual Interfaces (Castiglione della Pescaia, Grosseto, Italy) (AVI '18). Association for Computing Machinery, New York, NY, USA, Article 18, 8 pages. https://doi.org/10.1145/3206505.3206522
[17]
Vrishab Krishna, Yi Ding, Aiwen Xu, and Tobias Höllerer. 2019. Multimodal Biometric Authentication for VR/AR Using EEG and Eye Tracking. In Adjunct of the 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI '19). Association for Computing Machinery, New York, NY, USA, Article 6, 5 pages. https://doi.org/10.1145/3351529.3360655
[18]
Manu Kumar, Tal Garfinkel, Dan Boneh, and Terry Winograd. 2007. Reducing Shoulder-Surfing by Using Gaze-Based Password Entry. In Proceedings of the 3rd Symposium on Usable Privacy and Security (Pittsburgh, Pennsylvania, USA) (SOUPS '07). Association for Computing Machinery, New York, NY, USA, 13--19. https://doi.org/10.1145/1280680.1280683
[19]
Alexander Kupin, Benjamin Moeller, Yijun Jiang, Natasha Kholgade Banerjee, and Sean Banerjee. 2019. Task-Driven Biometric Authentication of Users in Virtual Reality (VR) Environments. In MultiMedia Modeling, Ioannis Kompatsiaris, Benoit Huet, Vasileios Mezaris, Cathal Gurrin, Wen-Huang Cheng, and Stefanos Vrochidis (Eds.). Springer International Publishing, Cham, 55--67.
[20]
Sukun Li, Sonal Savaliya, Leonard Marino, Avery M. Leider, and Charles C. Tappert. 2019. Brain Signal Authentication for Human-Computer Interaction in Virtual Reality. In 2019 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC). IEEE, New York, NY, USA, 115--120. https://doi.org/10.1109/CSE/EUC.2019.00031
[21]
Jonathan Liebers, Patrick Horn, Christian Burschik, Uwe Gruenefeld, and Stefan Schneegass. 2021. Using Gaze Behavior and Head Orientation for Implicit Identification in Virtual Reality. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology (Osaka, Japan) (VRST '21). Association for Computing Machinery, New York, NY, USA, Article 22, 9 pages. https://doi.org/10.1145/3489849.3489880
[22]
Lorraine Lin and Sophie Jörg. 2016. Need a Hand? How Appearance Affects the Virtual Hand Illusion. In Proceedings of the ACM Symposium on Applied Perception (Anaheim, California) (SAP '16). Association for Computing Machinery, New York, NY, USA, 69--76. https://doi.org/10.1145/2931002.2931006
[23]
Florian Mathis, Hassan Ismail Fawaz, and Mohamed Khamis. 2020. Knowledge-Driven Biometric Authentication in Virtual Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/3334480.3382799
[24]
Florian Mathis, John Williamson, Kami Vaniea, and Mohamed Khamis. 2020. RubikAuth: Fast and Secure Authentication in Virtual Reality. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3334480.3382827
[25]
Robert Miller, Natasha Kholgade Banerjee, and Sean Banerjee. 2020. Within-System and Cross-System Behavior-Based Biometric Authentication in Virtual Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, New York, NY, USA, 311--316. https://doi.org/10.1109/VRW50115.2020.00070
[26]
Tahrima Mustafa, Richard Matovu, Abdul Serwadda, and Nicholas Muirhead. 2018. Unsure How to Authenticate on Your VR Headset? Come on, Use Your Head!. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics (Tempe, AZ, USA) (IWSPA '18). Association for Computing Machinery, New York, NY, USA, 23--30. https://doi.org/10.1145/3180445.3180450
[27]
Ilesanmi Olade, Charles Fleming, and Hai-Ning Liang. 2020. BioMove: Biometric User Identification from Human Kinesiological Movements for Virtual Reality Systems. Sensors, Vol. 20, 10 (2020). https://doi.org/10.3390/s20102944
[28]
Ilesanmi Olade, Hai-Ning Liang, Charles Fleming, and Christopher Champion. 2020. Exploring the Vulnerabilities and Advantages of SWIPE or Pattern Authentication in Virtual Reality (VR). In Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality Simulations (Sydney, NSW, Australia) (ICVARS 2020). Association for Computing Machinery, New York, NY, USA, 45--52. https://doi.org/10.1145/3385378.3385385
[29]
Ken Pfeuffer, Matthias J. Geiger, Sarah Prange, Lukas Mecke, Daniel Buschek, and Florian Alt. 2019. Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300340
[30]
Dario Pittera, Elia Gatti, and Marianna Obrist. 2019. I'm Sensing in the Rain: Spatial Incongruity in Visual-Tactile Mid-Air Stimulation Can Elicit Ownership in VR Users. Association for Computing Machinery, New York, NY, USA, 1--15. https://doi.org/10.1145/3290605.3300362
[31]
Majed Samad, Elia Gatti, Anne Hermes, Hrvoje Benko, and Cesare Parise. 2019. Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3290605.3300550
[32]
Valentin Schwind, Lorraine Lin, Massimiliano Di Luca, Sophie Jörg, and James Hillis. 2018. Touch with Foreign Hands: The Effect of Virtual Hand Appearance on Visual-Haptic Integration. In Proceedings of the 15th ACM Symposium on Applied Perception (Vancouver, British Columbia, Canada) (SAP '18). Association for Computing Machinery, New York, NY, USA, Article 9, 8 pages. https://doi.org/10.1145/3225153.3225158
[33]
Yiran Shen, Hongkai Wen, Chengwen Luo, Weitao Xu, Tao Zhang, Wen Hu, and Daniela Rus. 2019. GaitLock: Protect Virtual and Augmented Reality Headsets Using Gait. IEEE Transactions on Dependable and Secure Computing, Vol. 16, 3 (2019), 484--497. https://doi.org/10.1109/TDSC.2018.2800048
[34]
Manimaran Sivasamy, V.N. Sastry, and N.P. Gopalan. 2020. VRCAuth: Continuous Authentication of Users in Virtual Reality Environment Using Head-Movement. In 2020 5th International Conference on Communication and Electronics Systems (ICCES). IEEE, New York, NY, USA, 518--523. https://doi.org/10.1109/ICCES48766.2020.9137914
[35]
Steeven Villa, Thomas Kosch, Felix Grelka, Albrecht Schmidt, and Robin Welsch. 2023. The placebo effect of human augmentation: Anticipating cognitive augmentation increases risk-taking behavior. Computers in Human Behavior, Vol. 146 (Sept. 2023), 107787. https://doi.org/10.1016/j.chb.2023.107787
[36]
Emanuel von Zezschwitz, Paul Dunphy, and Alexander De Luca. 2013. Patterns in the Wild: A Field Study of the Usability of Pattern and Pin-Based Authentication on Mobile Devices. In Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (Munich, Germany) (MobileHCI '13). Association for Computing Machinery, New York, NY, USA, 261--270. https://doi.org/10.1145/2493190.2493231
[37]
Yannick Weiss, Steeven Villa, Albrecht Schmidt, Sven Mayer, and Florian Müller. 2023. Using Pseudo-Stiffness to Enrich the Haptic Experience in Virtual Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 388, 15 pages. https://doi.org/10.1145/3544548.3581223
[38]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver BC Canada, 143--146. https://doi.org/10.1145/1978942.1978963
[39]
Zhen Yu, Hai-Ning Liang, Charles Fleming, and Ka Lok Man. 2016. An exploration of usable authentication mechanisms for virtual reality systems. In 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). IEEE, New York, NY, USA, 458--460. https://doi.org/10.1109/APCCAS.2016.7804002
[40]
Nur Haryani Zakaria, David Griffiths, Sacha Brostoff, and Jeff Yan. 2011. Shoulder Surfing Defence for Recall-Based Graphical Passwords. In Proceedings of the Seventh Symposium on Usable Privacy and Security (Pittsburgh, Pennsylvania) (SOUPS '11). Association for Computing Machinery, New York, NY, USA, Article 6, 12 pages. https://doi.org/10.1145/2078827.2078835
[41]
André Zenner, Hannah Maria Kriegler, and Antonio Krüger. 2021. HaRT - The Virtual Reality Hand Redirection Toolkit. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA '21). Association for Computing Machinery, New York, NY, USA, Article 387, 7 pages. https://doi.org/10.1145/3411763.3451814
[42]
André Zenner and Antonio Krüger. 2019. Estimating Detection Thresholds for Desktop-Scale Hand Redirection in Virtual Reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, New York, NY, USA, 47--55. https://doi.org/10.1109/VR.2019.8798143
[43]
Yongtuo Zhang, Wen Hu, Weitao Xu, Chun Tung Chou, and Jiankun Hu. 2018. Continuous Authentication Using Eye Movement Response of Implicit Visual Stimuli. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 1, 4, Article 177 (jan 2018), 22 pages. https://doi.org/10.1145/3161410
[44]
Huadi Zhu, Wenqiang Jin, Mingyan Xiao, Srinivasan Murali, and Ming Li. 2020. BlinKey: A Two-Factor User Authentication Method for Virtual Reality Devices. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, 4, Article 164 (dec 2020), 29 pages. https://doi.org/10.1145/3432217

Index Terms

  1. Exploring Redirection and Shifting Techniques to Mask Hand Movements from Shoulder-Surfing Attacks during PIN Authentication in Virtual Reality

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Human-Computer Interaction
      Proceedings of the ACM on Human-Computer Interaction  Volume 8, Issue MHCI
      MHCI
      September 2024
      1136 pages
      EISSN:2573-0142
      DOI:10.1145/3697825
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 24 September 2024
      Published in PACMHCI Volume 8, Issue MHCI

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. hand redirection
      2. shoulder-surfing
      3. virtual reality

      Qualifiers

      • Research-article

      Funding Sources

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 134
        Total Downloads
      • Downloads (Last 12 months)134
      • Downloads (Last 6 weeks)12
      Reflects downloads up to 23 Feb 2025

      Other Metrics

      Citations

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media