skip to main content
10.1145/3052973.3053032acmconferencesArticle/Chapter ViewAbstractPublication Pagesasia-ccsConference Proceedingsconference-collections
research-article

Evaluating Behavioral Biometrics for Continuous Authentication: Challenges and Metrics

Published: 02 April 2017 Publication History

Abstract

In recent years, behavioral biometrics have become a popular approach to support continuous authentication systems. Most generally, a continuous authentication system can make two types of errors: false rejects and false accepts. Based on this, the most commonly reported metrics to evaluate systems are the False Reject Rate (FRR) and False Accept Rate (FAR). However, most papers only report the mean of these measures with little attention paid to their distribution. This is problematic as systematic errors allow attackers to perpetually escape detection while random errors are less severe. Using 16 biometric datasets we show that these systematic errors are very common in the wild. We show that some biometrics (such as eye movements) are particularly prone to systematic errors, while others (such as touchscreen inputs) show more even error distributions. Our results also show that the inclusion of some distinctive features lowers average error rates but significantly increases the prevalence of systematic errors. As such, blind optimization of the mean EER (through feature engineering or selection) can sometimes lead to lower security. Following this result we propose the Gini Coefficient (GC) as an additional metric to accurately capture different error distributions. We demonstrate the usefulness of this measure both to compare different systems and to guide researchers during feature selection. In addition to the selection of features and classifiers, some non- functional machine learning methodologies also affect error rates. The most notable examples of this are the selection of training data and the attacker model used to develop the negative class. 13 out of the 25 papers we analyzed either include imposter data in the negative class or randomly sample training data from the entire dataset, with a further 6 not giving any information on the methodology used. Using real-world data we show that both of these decisions lead to significant underestimation of error rates by 63% and 81%, respectively. This is an alarming result, as it suggests that researchers are either unaware of the magnitude of these effects or might even be purposefully attempting to over-optimize their EER without actually improving the system.

References

[1]
A. A. E. Ahmed and I. Traore. A new biometric technology based on mouse dynamics. Dependable and Secure Computing, IEEE Transactions on, 4(3):165--179, 2007.
[2]
H. J. Ailisto, M. Lindholm, J. Mantyjarvi, E. Vildjiounaite, and S.-M. Makela. Identifying people from gait pattern with accelerometers. In Defense and Security, pages 7--14. International Society for Optics and Photonics, 2005.
[3]
K. Allix, T. F. Bissyandé, J. Klein, and Y. Le Traon. Are your training datasets yet relevant? In International Symposium on Engineering Secure Software and Systems, pages 51--67. Springer, 2015.
[4]
K. Allix, T. F. D. A. Bissyande, J. Klein, and Y. Le Traon. Machine learning-based malware detection for android applications: History matters! Technical report, University of Luxembourg, SnT, 2014.
[5]
S. Axelsson. The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security (TISSEC), 3(3):186--205, 2000.
[6]
P. Bours and S. Mondal. Performance evaluation of continuous authentication systems. IET Biometrics, 4(4):220--226, 2015.
[7]
A. Brajdic and R. Harle. Walk detection and step counting on unconstrained smartphones. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and ubiquitous computing, pages 225--234. ACM, 2013.
[8]
Ş. Budulan, E. Burceanu, T. Rebedea, and C. Chiru. Continuous user authentication using machine learning on touch dynamics. In International Conference on Neural Information Processing, pages 591--598. Springer, 2015.
[9]
D. Buschek, A. De Luca, and F. Alt. Improving accuracy, applicability and usability of keystroke biometrics on mobile touchscreen devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 1393--1402. ACM, 2015.
[10]
Z. Cai, C. Shen, M. Wang, Y. Song, and J. Wang. Mobile authentication through touch-behavior features. In Biometric Recognition, pages 386--393. Springer, 2013.
[11]
M. O. Derawi, C. Nickel, P. Bours, and C. Busch. Unobtrusive user-authentication on mobile phones using biometric gait recognition. In Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), 2010 Sixth International Conference on, pages 306--311. IEEE, 2010.
[12]
B. Draffin, J. Zhu, and J. Zhang. Keysens: Passive user authentication through micro-behavior modeling of soft keyboard interaction. In International Conference on Mobile Computing, Applications, and Services, pages 184--201. Springer, 2013.
[13]
S. Eberz, K. B. Rasmussen, V. Lenders, and I. Martinovic. Preventing lunchtime attacks: Fighting insider threats with eye movement biometrics. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS), 2015.
[14]
S. Eberz, K. B. Rasmussen, V. Lenders, and I. Martinovic. Looks like eve: Exposing insider threats using eye movement biometrics. ACM Transactions on Privacy and Security, 19(1):1, 2016.
[15]
T. Feng, J. Yang, Z. Yan, E. M. Tapia, and W. Shi. Tips: Context-aware implicit user identification using touch screen in uncontrolled environments. In Proceedings of the 15th Workshop on Mobile Computing Systems and Applications, page 9. ACM, 2014.
[16]
M. Frank, R. Biedert, E. Ma, I. Martinovic, and D. Song. Touchalytics: On the applicability of touchscreen input as a behavioral biometric for continuous authentication. Information Forensics and Security, IEEE Transactions on, 8(1):136--148, 2013.
[17]
D. Gafurov, K. Helkala, and T. Søndrol. Biometric gait authentication using accelerometer sensor. Journal of computers, 1(7):51--59, 2006.
[18]
H. Gascon, S. Uellenbeck, C. Wolf, and K. Rieck. Continuous authentication on mobile devices by analysis of typing motion behavior. In Sicherheit, pages 1--12. Citeseer, 2014.
[19]
C. Gini. Variabilità e mutabilità. Reprinted in Memorie di metodologica statistica (Ed. Pizetti E, Salvemini, T). Rome: Libreria Eredi Virgilio Veschi, 1, 1912.
[20]
M. Goffredo, I. Bouchrika, J. N. Carter, and M. S. Nixon. Self-calibrating view-invariant gait biometrics. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 40(4):997--1008, 2010.
[21]
Z. Jorgensen and T. Yu. On mouse dynamics as a behavioral biometric for authentication. In Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, pages 476--482. ACM, 2011.
[22]
T. Kinnunen, F. Sedlak, and R. Bednarik. Towards task-independent person authentication using eye movement signals. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, pages 187--190. ACM, 2010.
[23]
J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S.-M. Makela, and H. Ailisto. Identifying users of portable devices from gait pattern with accelerometers. In Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP'05). IEEE International Conference on, volume 2, pages ii--973. IEEE, 2005.
[24]
S. Mondal and P. Bours. Continuous authentication using mouse dynamics. In Biometrics Special Interest Group (BIOSIG), 2013 International Conference of the, pages 1--12. IEEE, 2013.
[25]
M. Pusara and C. E. Brodley. User re-authentication via mouse movements. In Proceedings of the 2004 ACM workshop on Visualization and data mining for computer security, pages 1--8. ACM, 2004.
[26]
K. B. Rasmussen, M. Roeschlin, I. Martinovic, and G. Tsudik. Authentication using pulse-response biometrics. In NDSS, 2014.
[27]
L. Rong, D. Zhiguo, Z. Jianzhong, and L. Ming. Identification of individual walking patterns using gait acceleration. In 2007 1st International Conference on Bioinformatics and Biomedical Engineering, pages 543--546. IEEE, 2007.
[28]
A. Roy, T. Halevi, and N. Memon. An hmm-based behavior modeling approach for continuous mobile authentication. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3789--3793. IEEE, 2014.
[29]
P. Saravanan, S. Clarke, D. H. P. Chau, and H. Zha. Latentgesture: active user authentication through background touch analysis. In Proceedings of the Second International Symposium of Chinese CHI, pages 110--113. ACM, 2014.
[30]
D. A. Schulz. Mouse curve biometrics. In 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference, pages 1--6. IEEE, 2006.
[31]
C. Shen, Y. Zhang, Z. Cai, T. Yu, and X. Guan. Touch-interaction behavior for continuous user authentication on smartphones. In 2015 International Conference on Biometrics (ICB), pages 157--162. IEEE, 2015.
[32]
M. Soriano, A. Araullo, and C. Saloma. Curve spreads-a biometric from front-view gait video. Pattern Recognition Letters, 25(14):1595--1602, 2004.
[33]
E. Vildjiounaite, S.-M. Makela, M. Lindholm, R. Riihimaki, V. Kyllönen, J. Mantyjarvi, and H. Ailisto. Unobtrusive multimodal biometrics for ensuring privacy and information security with personal devices. In International Conference on Pervasive Computing, pages 187--201. Springer, 2006.
[34]
A. Weiss, A. Ramapanicker, P. Shah, S. Noble, and L. Immohr. Mouse movements biometric identification: A feasibility study. Proc. Student/Faculty Research Day CSIS, Pace University, White Plains, NY, 2007.
[35]
H. Xu, Y. Zhou, and M. R. Lyu. Towards continuous and passive authentication via touch biometrics: An experimental study on smartphones. In Symposium On Usable Privacy and Security (SOUPS 2014), pages 187--198, 2014.
[36]
X. Zhao, T. Feng, and W. Shi. Continuous mobile authentication using a novel graphic touch gesture feature. In Biometrics: Theory, Applications and Systems (BTAS), 2013 IEEE Sixth International Conference on, pages 1--6. IEEE, 2013.
[37]
N. Zheng, A. Paloski, and H. Wang. An efficient user verification system via mouse movements. In Proceedings of the 18th ACM conference on Computer and communications security, pages 139--150. ACM, 2011.

Cited By

View all
  • (2024)De-Anonymizing Avatars in Virtual Reality: Attacks and CountermeasuresIEEE Transactions on Mobile Computing10.1109/TMC.2024.342604623:12(13342-13357)Online publication date: Dec-2024
  • (2024)A Novel Evaluation Framework for Biometric Security: Assessing Guessing Difficulty as a MetricIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.345593019(8369-8384)Online publication date: 2024
  • (2024)SSPRA: A Robust Approach to Continuous Authentication Amidst Real-World Adversarial ChallengesIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.33695906:2(245-260)Online publication date: Apr-2024
  • Show More Cited By

Index Terms

  1. Evaluating Behavioral Biometrics for Continuous Authentication: Challenges and Metrics

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASIA CCS '17: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security
    April 2017
    952 pages
    ISBN:9781450349444
    DOI:10.1145/3052973
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 April 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. biometrics
    2. continuous authentication
    3. metrics

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ASIA CCS '17
    Sponsor:

    Acceptance Rates

    ASIA CCS '17 Paper Acceptance Rate 67 of 359 submissions, 19%;
    Overall Acceptance Rate 418 of 2,322 submissions, 18%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)170
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)De-Anonymizing Avatars in Virtual Reality: Attacks and CountermeasuresIEEE Transactions on Mobile Computing10.1109/TMC.2024.342604623:12(13342-13357)Online publication date: Dec-2024
    • (2024)A Novel Evaluation Framework for Biometric Security: Assessing Guessing Difficulty as a MetricIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.345593019(8369-8384)Online publication date: 2024
    • (2024)SSPRA: A Robust Approach to Continuous Authentication Amidst Real-World Adversarial ChallengesIEEE Transactions on Biometrics, Behavior, and Identity Science10.1109/TBIOM.2024.33695906:2(245-260)Online publication date: Apr-2024
    • (2024)Enhancing smartphone security with human centric bimodal fallback authentication leveraging sensorsScientific Reports10.1038/s41598-024-74473-714:1Online publication date: 21-Oct-2024
    • (2024)NeuroIDBench: An open-source benchmark framework for the standardization of methodology in brainwave-based authentication researchJournal of Information Security and Applications10.1016/j.jisa.2024.10383285(103832)Online publication date: Sep-2024
    • (2024)SHRIMPS: A framework for evaluating multi-user, multi-modal implicit authentication systemsComputers & Security10.1016/j.cose.2023.103594137(103594)Online publication date: Feb-2024
    • (2024)Continuous Authentication with Eye Movement BiometricsProceedings of the Eighth International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’24), Volume 110.1007/978-3-031-77688-5_35(369-377)Online publication date: 20-Dec-2024
    • (2023)Analyzing Cyber Security Research Practices through a Meta-Research FrameworkProceedings of the 16th Cyber Security Experimentation and Test Workshop10.1145/3607505.3607523(64-74)Online publication date: 7-Aug-2023
    • (2023)How Unique do we Move? Understanding the Human Body and Context Factors for User IdentificationProceedings of Mensch und Computer 202310.1145/3603555.3603574(127-137)Online publication date: 3-Sep-2023
    • (2023)Revisiting the Security of Biometric Authentication Systems Against Statistical AttacksACM Transactions on Privacy and Security10.1145/357174326:2(1-30)Online publication date: 12-Apr-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media