abstract

Knowledge-driven Biometric Authentication in Virtual Reality

Authors:

Florian Mathis,

Hassan Ismail Fawaz,

Mohamed KhamisAuthors Info & Claims

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 10

https://doi.org/10.1145/3334480.3382799

Published: 25 April 2020 Publication History

Abstract

With the increasing adoption of virtual reality (VR) in public spaces, protecting users from observation attacks is becoming essential to prevent attackers from accessing context-sensitive data or performing malicious payment transactions in VR. In this work, we propose RubikBiom, a knowledge-driven behavioural biometric authentication scheme for authentication in VR. We show that hand movement patterns performed during interactions with a knowledge-based authentication scheme (e.g., when entering a PIN) can be leveraged to establish an additional security layer. Based on a dataset gathered in a lab study with 23 participants, we show that knowledge-driven behavioural biometric authentication increases security in an unobtrusive way. We achieve an accuracy of up to 98.91% by applying a Fully Convolutional Network (FCN) on 32 authentications per subject. Our results pave the way for further investigations towards knowledge-driven behavioural biometric authentication in VR.

References

[1]

Israa Alsaadi. 2015. Physiological Biometric Authentication Systems, Advantages, Disadvantages And Future Development: A Review. International Journal of Scientific Technology Research Volume 4 (12 2015).

[2]

Ajit Ashwin, Kohlgade Baneerjee Natasha, and Banerjee Sean. 2019. Combining Pairwise Feature Matches from Device Trajectories for Biometric Authentication in Virtual Reality Environments. (2019).

[3]

Yoshua Bengio. 2012. Practical Recommendations for Gradient-Based Training of Deep Architectures. Springer Berlin Heidelberg, Berlin, Heidelberg, 437--478.

[4]

Rasekhar Bhagavatula, Blase Ur, Kevin Iacovino, Su Mon Kywe, Lorrie Faith Cranor, and Marios Savvides. 2015. Biometric authentication on iphone and android: Usability, perceptions, and influences on adoption. (2015).

[5]

Debnath Bhattacharyya, Rahul Ranjan, Farkhod Alisherov, Minkyu Choi, and others. 2009. Biometric authentication: A review. International Journal of u-and e-Service, Science and Technology 2, 3 (2009), 13--28.

[6]

William Buxton and Brad Myers. 1986. A Study in Two-handed Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '86). ACM, New York, NY, USA, 321--326.

Digital Library

[7]

François Chollet and others. 2015. Keras. https://keras.io. (2015).

[8]

Sauvik Das, David Lu, Taehoon Lee, Joanne Lo, and Jason I. Hong. 2019. The Memory Palace: Exploring Visual-Spatial Paths for Strong, Memorable, Infrequent Authentication. In Proceedings of the 32Nd Annual ACM Symposium on User Interface Software and Technology (UIST '19). ACM, New York, NY, USA, 1109--1121.

Digital Library

[9]

Alexander De Luca, Marian Harbach, Emanuel von Zezschwitz, Max-Emanuel Maurer, Bernhard Ewald Slawik, Heinrich Hussmann, and Matthew Smith. 2014. Now You See Me, Now You Don'T: Protecting Smartphone Authentication from Shoulder Surfers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2937--2946.

Digital Library

[10]

Malin Eiband, Mohamed Khamis, Emanuel Von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. Understanding shoulder surfing in the wild: Stories from users and observers. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4254--4265.

Digital Library

[11]

Philippe Esling and Carlos Agon. 2012. Time-series Data Mining. ACM Comput. Surv. 45, 1, Article 12 (Dec. 2012), 34 pages.

Digital Library

[12]

Markus Funk, Karola Marky, Iori Mizutani, Mareike Kritzler, Simon Mayer, and Florian Michahelles. 2019. LookUnlock: Using Spatial-Targets for User-Authentication on HMDs. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, New York, NY, USA, Article LBW0114, 6 pages.

Digital Library

[13]

Simson Garfinkel and Heather Richter Lipford. 2014. Usable security: History, themes, and challenges. Synthesis Lectures on Information Security, Privacy, and Trust 5, 2 (2014), 1--124.

[14]

Ceenu George, Philipp Janssen, David Heuss, and Florian Alt. 2019a. Should I Interrupt or Not?: Understanding Interruptions in Head-Mounted Display Settings. In Proceedings of the 2019 on Designing Interactive Systems Conference. ACM, 497--510.

Digital Library

[15]

Ceenu George, Mohamed Khamis, Daniel Buschek, and Heinrich Hussmann. 2019b. Investigating the Third Dimension for Authentication in Immersive Virtual Reality and in the Real World. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 277--285.

[16]

Ceenu George, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017a. Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality. In Network and Distributed System Security Symposium (NDSS 2017) (USEC '17). NDSS.

[17]

Ceenu George, Mohamed Khamis, Emanuel von Zezschwitz, Marinus Burger, Henri Schmidt, Florian Alt, and Heinrich Hussmann. 2017b. Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality. In Network and Distributed System Security Symposium (NDSS 2017) (USEC '17). NDSS.

[18]

Jan Gugenheimer, Christian Mai, Mark McGill, Julie Williamson, Frank Steinicke, and Ken Perlin. 2019. Challenges Using Head-Mounted Displays in Shared and Social Spaces. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, New York, NY, USA, Article W19, 8 pages.

Digital Library

[19]

Yves Guiard. 1987. Asymmetric Division of Labor in Human Skilled Bimanual Action. Journal of Motor Behavior 19, 4 (1987), 486--517. 15136274.

[20]

INQUIRER.net. 2016. Alibaba launches full VR shopping experience with Buy+. (2016). https://technology.inquirer.net/56131/alibaba-launches-full-vr-shopping-experience-buy accessed 29 August 2019.

[21]

Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. 2019. Deep learning for time series classification: A review. Data Mining and Knowledge Discovery 33, 4 (01 Jul 2019), 917--963.

[22]

Paul Kabbash, William Buxton, and Abigail Sellen. 1994. Two-handed input in a compound task. In CHI, Vol. 94. 417--423.

[23]

Christina Katsini, Marios Belk, Christos Fidas, Nikolaos Avouris, and George Samaras. 2016. Security and Usability in Knowledge-based User Authentication: A Review. In Proceedings of the 20th Pan-Hellenic Conference on Informatics (PCI '16). ACM, New York, NY, USA, Article 63, 6 pages.

Digital Library

[24]

Abinaya Kumar, Aishwarya Radjesh, Sven Mayer, and Huy Viet Le. 2019. Improving the Input Accuracy of Touchscreens Using Deep Learning. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, New York, NY, USA, Article LBW1514, 6 pages.

Digital Library

[25]

Alexander Kupin, Benjamin Moeller, Yijun Jiang, Natasha Kholgade Banerjee, and Sean Banerjee. 2019. Task-Driven Biometric Authentication of Users in Virtual Reality (VR) Environments. In MultiMedia Modeling, Ioannis Kompatsiaris, Benoit Huet, Vasileios Mezaris, Cathal Gurrin, Wen-Huang Cheng, and Stefanos Vrochidis (Eds.). Springer International Publishing, Cham, 55--67.

[26]

Václav Matyá and Zdenek-Ríha. 2002. Biometric Authentication - Security and Usability. Springer US, Boston, MA, 227--239.

[27]

Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A dose of reality: Overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2143--2152.

Digital Library

[28]

Tahrima Mustafa, Richard Matovu, Abdul Serwadda, and Nicholas Muirhead. 2018. Unsure How to Authenticate on Your VR Headset?: Come on, Use Your Head!. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics (IWSPA '18). ACM, New York, NY, USA, 23--30.

Digital Library

[29]

Ken Pfeuffer, Matthias J. Geiger, Sarah Prange, Lukas Mecke, Daniel Buschek, and Florian Alt. 2019. Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 110, 12 pages.

Digital Library

[30]

M. A. Sasse, S. Brostoff, and D. Weirich. 2001. Transforming the 'Weakest Link' - a Human/Computer Interaction Approach to Usable and Effective Security. BT Technology Journal 19, 3 (01 Jul 2001), 122--131.

Digital Library

[31]

Y. Shen, H. Wen, C. Luo, W. Xu, T. Zhang, W. Hu, and D. Rus. 2019. GaitLock: Protect Virtual and Augmented Reality Headsets Using Gait. IEEE Transactions on Dependable and Secure Computing 16, 3 (May 2019), 484--497.

[32]

Facebook Technologies. 2019a. Oculus Quest All-in-One-VR. (2019). https://www.oculus.com/quest/

[33]

Unity Technologies. 2019b. Quaternions in Unity 3D. (2019). https://docs.unity3d.com/Manual/QuaternionAndEulerRotationsInUnity.html

[34]

Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2019. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters 119 (2019), 3 -- 11. //doi.org/10.1016/j.patrec.2018.02.010 Deep Learning for Pattern Recognition.

Digital Library

[35]

Z. Wang, W. Yan, and T. Oates. 2017. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International Joint Conference on Neural Networks (IJCNN). 1578--1585.

[36]

Julie R. Williamson, Mark McGill, and Khari Outram. 2019. PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 80, 14 pages.

Digital Library

[37]

QIANG YANG and XINDONG WU. 2006. 10 CHALLENGING PROBLEMS IN DATA MINING RESEARCH. International Journal of Information Technology & Decision Making 05, 04 (2006), 597--604.

[38]

Shuochao Yao, Shaohan Hu, Yiran Zhao, Aston Zhang, and Tarek Abdelzaher. 2017. DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing. In Proceedings of the 26th International Conference on World Wide Web (WWW '17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 351--360.

Digital Library

[39]

S. Yi, Z. Qin, E. Novak, Y. Yin, and Q. Li. 2016. GlassGesture: Exploring head gesture interface of smart glasses. In IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications. 1--9.

Digital Library

[40]

Z. Yu, H. Liang, C. Fleming, and K. L. Man. 2016. An exploration of usable authentication mechanisms for virtual reality systems. In 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS). 458--460.

Cited By

Xiao GLing ZFan QXu XWu WDing DChen CFu X(2025)Pivot: Panoramic-Image-Based VR User Authentication against Side-Channel AttacksACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369497521:2(1-19)Online publication date: 2-Jan-2025
https://dl.acm.org/doi/10.1145/3694975
Yang KZhang ZTian YMa J(2025)A Traceable Continuous Authentication Scheme for Avatars in Large-Scale Commercial ActivitiesIEEE Internet of Things Journal10.1109/JIOT.2024.348692412:5(5194-5206)Online publication date: 1-Mar-2025
https://doi.org/10.1109/JIOT.2024.3486924
Li MBanerjee NBanerjee S(2025)Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR63409.2025.00043(223-230)Online publication date: 27-Jan-2025
https://doi.org/10.1109/AIxVR63409.2025.00043
Show More Cited By

Index Terms

Knowledge-driven Biometric Authentication in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Security and privacy
  1. Security services
    1. Authentication

Recommendations

Partial palm vein based biometric authentication
Abstract
Palm vein biometrics is a potential authentication technique. In addition to the inherent liveness property, the palm vein pattern exhibits high authentication accuracy and anti-spoofing capability due to the dense vein pattern and broad coverage ...
Multi-factor biometrics for authentication: a false sense of security
MM&Sec '10: Proceedings of the 12th ACM workshop on Multimedia and security

Multi-factor biometric authentications have been proposed recently to strengthen security and/or privacy of biometric systems in addition to enhancing authentication accuracy. An important approach to multi-factor biometric authentication is to apply ...
Biometric Authentication: A Trustworthy Technology for Improved Authentication
ICFN '09: Proceedings of the 2009 International Conference on Future Networks

Authentication plays a vital role in various applications. Even though a number of effective methods and techniques are available, new ideas and techniques are necessary to overcome some of the limitations of these systems. At the present time, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

4474 pages

ISBN:9781450368193

DOI:10.1145/3334480

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2020

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

University of Edinburgh and the University of Glasgow jointly funded PhD studentships
Royal Society of Edinburgh

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
912
Total Downloads

Downloads (Last 12 months)163
Downloads (Last 6 weeks)12

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xiao GLing ZFan QXu XWu WDing DChen CFu X(2025)Pivot: Panoramic-Image-Based VR User Authentication against Side-Channel AttacksACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369497521:2(1-19)Online publication date: 2-Jan-2025
https://dl.acm.org/doi/10.1145/3694975
Yang KZhang ZTian YMa J(2025)A Traceable Continuous Authentication Scheme for Avatars in Large-Scale Commercial ActivitiesIEEE Internet of Things Journal10.1109/JIOT.2024.348692412:5(5194-5206)Online publication date: 1-Mar-2025
https://doi.org/10.1109/JIOT.2024.3486924
Li MBanerjee NBanerjee S(2025)Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR)10.1109/AIxVR63409.2025.00043(223-230)Online publication date: 27-Jan-2025
https://doi.org/10.1109/AIxVR63409.2025.00043
Hallal LRhinelander JVenkat R(2024)Recent Trends of Authentication Methods in Extended Reality: A SurveyApplied System Innovation10.3390/asi70300457:3(45)Online publication date: 28-May-2024
https://doi.org/10.3390/asi7030045
Jiao ADuan DXu W(2024)Medusa3D: The Watchful Eye Freezing Illegitimate Users in Virtual Reality InteractionsProceedings of the ACM on Human-Computer Interaction10.1145/36765158:MHCI(1-21)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676515
Weiss YVilla SGrootjen JHoppe MKale YMüller F(2024)Exploring Redirection and Shifting Techniques to Mask Hand Movements from Shoulder-Surfing Attacks during PIN Authentication in Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36765028:MHCI(1-24)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676502
Cheng RWu YKundu ALatapie HLee MChen SHan BShu YLiu JTan RHe YChen J(2024)MetaFL: Privacy-preserving User Authentication in Virtual Reality with Federated LearningProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699322(54-67)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699322
Shao WLuo SYan ZGanesan DLane NShi W(2024)Cross-content User Authentication in Virtual RealityProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3696212(2098-2105)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3696212
Rupp DGrießer PBonsch AKuhlen T(2024)Authentication in Immersive Virtual Environments through Gesture-Based Interaction with a Virtual Agent2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00016(54-60)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00016
Zhang ZYang KTian YMa J(2024)An Anti-Disguise Authentication System Using the First Impression of Avatar in MetaverseIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341052719(6393-6408)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3410527
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten