skip to main content
10.1145/3243734.3243769acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article

On the Accuracy of Password Strength Meters

Published: 15 October 2018 Publication History

Abstract

Password strength meters are an important tool to help users choose secure passwords. Strength meters can only then provide reasonable guidance when they are accurate, i.e., their score correctly reflect password strength. A strength meter with low accuracy may do more harm than good and guide the user to choose passwords with a high score but low actual security. While a substantial number of different strength meters is proposed in the literature and deployed in practice, we are lacking a clear picture of which strength meters provide high accuracy, and thus are most helpful for guiding users. Furthermore, we lack a clear understanding of how to compare accuracies of strength meters. In this work, (i) we propose a set of properties that a strength meter needs to fulfill to be considered to have high accuracy, (ii) we use these properties to select a suitable measure that can determine the accuracy of strength meters, and (iii) we use the selected measure to compare a wide range of strength meters proposed in the academic literature, provided by password managers, operating systems, and those used on websites. We expect our work to be helpful in the selection of good password strength meters by service operators, and to aid the further development of improved strength meters.

Supplementary Material

MP4 File (p1567-golla.mp4)

References

[1]
8bit Solutions, LLC. 2018. bitwarden (Web) -- Free Open Source Password Manager. https://bitwarden.com, as of today.
[2]
AgileBits, Inc. 2018. 1Password (Web) -- Password Manager. https://1password.com, as of today.
[3]
Fabian Angelstorf and Franziska Juckel. 2017. OMEN v0.3.0 - C Implementation of a Markov Model-based Password Guesser. https://github.com/RUB-SysSec/OMEN, as of today.
[4]
Daniel V. Bailey, Markus Dürmuth, and Christof Paar. 2014. Statistics on Password Re-use and Adaptive Strength for Financial Accounts Security and Cryptography for Networks (SCN '14). Springer, Amalfi, Italy, 218--235.
[5]
Adam Beautement, M. Angela Sasse, and Mike Wonham. 2008. The Compliance Budget: Managing Security Behaviour in Organisations New Security Paradigms Workshop (NSPW '08). ACM, Lake Tahoe, California, USA, 47--58.
[6]
Sebastian Benvenuti. 2012. Ubiquity -- Ubuntu Should Encourage Stronger Passwords. https://bugs.launchpad.net/ubuntu/source/ubiquity/bug/1044868, as of today.
[7]
Matt Bishop and Daniel V. Klein. 1995. Improving System Security via Proactive Password Checking. Computers & Security Vol. 14, 3 (1995), 233--249.
[8]
Joseph Bonneau. 2012. The Science of Guessing: Analyzing an Anonymized Corpus of 70 Million Passwords IEEE Symposium on Security and Privacy (SP '12). IEEE Computer Society, San Francisco, California, USA, 538--552.
[9]
Joseph Bonneau, Cormac Herley, Paul C. van Oorschot, and Frank Stajano. 2012. The Quest to Replace Passwords: A Framework for Comparative Evaluation of Web Authentication Schemes. In IEEE Symposium on Security and Privacy (SP '12). IEEE Computer Society, San Jose, California, USA, 553--567.
[10]
Mark Burnett. 2015. Today I Am Releasing Ten Million Passwords. https://xato.net/today-i-am-releasing-ten-million-passwords-b6278bbe7495, as of today.
[11]
William E. Burr, Donna F. Dodson, and W. Timothy Polk. 2004. Electronic Authentication Guideline: NIST SP 800--63 Ver. 1.0 (2004) to 800--63--2 (2013). https://csrc.nist.gov/publications/detail/sp/800--63/ver-10/archive/2004-06--30, as of today.
[12]
Javier Carranza and Contributors. 2018. Ubiquity -- Ubuntu Live CD Installer. https://launchpad.net/ubuntu/source/ubiquity, as of today.
[13]
Claude Castelluccia, Markus Dürmuth, and Daniele Perito. 2012. Adaptive Password-Strength Meters from Markov Models Symposium on Network and Distributed System Security (NDSS '12). The Internet Society, San Diego, California, USA.
[14]
Dashlane, Inc. 2018. Dashlane (Windows) -- Password Manager. https://www.dashlane.com, as of today.
[15]
"dcopi". 2013. NIST - Password Strength Meter Example. https://github.com/dcopi/PWStrength, as of today.
[16]
Xavier de Carné de Carnavalet and Mohammad Mannan. 2014 a. From Very Weak to Very Strong: Analyzing Password-Strength Meters Symposium on Network and Distributed System Security (NDSS '14). ISOC, San Diego, California, USA.
[17]
Xavier de Carné de Carnavalet and Mohammad Mannan. 2014 b. Password Multi-Checker Tool. https://madiba.encs.concordia.ca/software/passwordchecker/, as of today.
[18]
Matteo Dell'Amico and Maurizio Filippone. 2015. Monte Carlo Strength Evaluation: Fast and Reliable Password Checking ACM Conference on Computer and Communications Security (CCS '15). ACM, Denver, Colorado, USA, 158--169.
[19]
Matteo Dell'Amico, Pietro Michiardi, and Yves Roudier. 2010. Password Strength: An Empirical Analysis. In Conference on Information Communications (INFOCOM '10). IEEE, San Diego, California, USA, 983--991.
[20]
Dropbox, Inc. and Contributors. 2017. zxcvbn v4.4.2 -- JavaScript Implementation of the zxcvbn Strength Meter. https://github.com/dropbox/zxcvbn, as of today.
[21]
Markus Dürmuth, Fabian Angelstorf, Claude Castelluccia, Daniele Perito, and Abdelberi Chaabane. 2015. OMEN: Faster Password Guessing Using an Ordered Markov Enumerator International Symposium on Engineering Secure Software and Systems (ESSoS '15). Springer, Milan, Italy, 119--132.
[22]
Serge Egelman, Andreas Sotirakopoulos, Ildar Muslukhov, Konstantin Beznosov, and Cormac Herley. 2013. Does My Password Go Up to Eleven?: The Impact of Password Meters on Password Selection ACM Conference on Human Factors in Computing Systems (CHI '13). ACM, Paris, France, 2379--2388.
[23]
Dinei Florencio and Cormac Herley. 2007. A Large-scale Study of Web Password Habits. In Conference on World Wide Web (WWW '07). ACM, Banff, Alberta, Canada, 657--666.
[24]
Dinei Florêncio, Cormac Herley, and Paul C. van Oorschot. 2014. An Administrator's Guide to Internet Password Research Large Installation System Administration Conference (LISA '14). USENIX, Seattle, Washington, USA, 44--61.
[25]
Dinei Florêncio, Cormac Herley, and Paul C. van Oorschot. 2014. Password Portfolios and the Finite-Effort User: Sustainably Managing Large Numbers of Accounts. In USENIX Security Symposium (SSYM '14). USENIX, San Diego, California, USA, 575--590.
[26]
Dinei Florêncio, Cormac Herley, and Paul C. van Oorschot. 2016. Pushing on String: The "Don't Care" Region of Password Strength. Commun. ACM Vol. 59, 11 (Oct. 2016), 66--74.
[27]
Maximilian Golla, Benedict Beuscher, and Markus Dürmuth. 2016. On the Security of Cracking-Resistant Password Vaults ACM Conference on Computer and Communications Security (CCS '16). ACM, Vienna, Austria, 1230--1241.
[28]
Maximilian Golla, Theodor Schnitzler, and Markus Dürmuth. 2018 a. "Will Any Password Do?" Exploring Rate-Limiting on the Web Who Are You?! Adventures in Authentication Workshop (WAY '18). USENIX, Baltimore, Maryland, USA.
[29]
Maximilian Golla, Ibrahim Sertkaya, and Markus Dürmuth. 2018 b. Password Strength Meter Comparison Website. https://password-meter-comparison.org, as of today.
[30]
Jeremi M. Gosney. 2017. Nvidia GTX 1080 Ti Hashcat Benchmarks. https://gist.github.com/epixoip/ace60d09981be09544fdd35005051505, as of today.
[31]
Paul A. Grassi, James L. Fenton, and William E. Burr. 2017. Digital Identity Guidelines -- Authentication and Lifecycle Management: NIST SP 800--63B (2017).
[32]
Yimin Guo and Zhenfeng Zhang. 2018. LPSE: Lightweight Password-Strength Estimation for Password Meters. Computers & Security Vol. 73 (March. 2018), 507--518.
[33]
Hana Habib, Jessica Colnago, William Melicher, Blase Ur, Sean Segreti, Lujo Bauer, Nicolas Christin, and Lorrie Cranor. 2017. Password Creation in the Presence of Blacklists. In Workshop on Usable Security (USEC '17). Internet Society, San Diego, California, USA.
[34]
Shiva Houshmand and Sudhir Aggarwal. 2012. Building Better Passwords Using Probabilistic Techniques Annual Computer Security Applications Conference (ACSAC '12). ACM, Orlando, Florida, USA, 109--118.
[35]
Jason Huggins and SeleniumHQ Contributors. 2017. Selenium - Web Browser Automation. http://www.seleniumhq.org, as of today.
[36]
Troy Hunt. 2018. 500m Pwned Passwords List. https://haveibeenpwned.com/Passwords, as of today.
[37]
Markus Jakobsson and Mayank Dhiman. 2012. The Benefits of Understanding Passwords. In USENIX Workshop on Hot Topics in Security (HotSec '12). USENIX, Bellevue, Washington, USA.
[38]
JS Foundation. 2018. Appium -- Automation Made Awesome. http://appium.io, as of today.
[39]
Keeper Security, Inc. 2018. Keeper (Web) -- Password Manager. https://keepersecurity.com, as of today.
[40]
Patrick Kelley, Saranga Kom, Michelle L. Mazurek, Rich Shay, Tim Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, and Julio López. 2012. Guess Again (and Again and Again): Measuring Password Strength by Simulating Password-Cracking Algorithms. In IEEE Symposium on Security and Privacy (SP '12). IEEE Computer Society, San Jose, California, USA, 523--537.
[41]
Daniel V. Klein. 1990. "Foiling the Cracker": A Survey of, and Improvements to, Password Security USENIX Security Workshop. USENIX, Berkeley, California, USA, 5--14.
[42]
Saranga Komanduri, Richard Shay, Patrick Gage Kelley, Michelle L. Mazurek, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, and Serge Egelman. 2011. Of Passwords and People: Measuring the Effect of Password-Composition Policies ACM Conference on Human Factors in Computing Systems (CHI '11). ACM, Vancouver, British Columbia, Canada, 2595--2604.
[43]
LogMeIn, Inc. 2018. LastPass (Web) -- Password Manager. https://www.lastpass.com, as of today.
[44]
Jerry Ma, Weining Yang, Min Luo, and Ninghui Li. 2014. A Study of Probabilistic Password Models. In IEEE Symposium on Security and Privacy (SP '14). IEEE, San Jose, CA, USA, 689--704.
[45]
William Melicher. 2017. Source Code -- Cracking Passwords with Neural Networks. https://github.com/cupslab/neural_network_cracking, as of today.
[46]
William Melicher, Blase Ur, Sean M. Segreti, Saranga Komanduri, Lujo Bauer, Nicolas Christin, and Lorrie Faith Cranor. 2016. Fast, Lean, and Accurate: Modeling Password Guessability Using Neural Networks USENIX Security Symposium (SSYM '16). USENIX, Austin, Texas, USA, 175--191.
[47]
Robert Morris and Ken Thompson. 1979. Password Security: A Case History. Commun. ACM Vol. 22, 11 (1979), 594--597.
[48]
Arvind Narayanan and Vitaly Shmatikov. 2005. Fast Dictionary Attacks on Passwords Using Time-Space Tradeoff ACM Conference on Computer and Communications Security (CCS '05). ACM, Alexandria, VA, USA, 364--372.
[49]
Ronald Oussoren. 2018. PyObjC -- The Python Objective-C Bridge. https://pythonhosted.org/pyobjc/, as of today.
[50]
Sarah Pearman, Jeremy Thomas, Pardis Emami Naeini, Hana Habib, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Serge Egelman, and Alain Forget. 2017. Let's Go in for a Closer Look: Observing Passwords in Their Natural Habitat ACM Conference on Computer and Communications Security (CCS '17). ACM, Dallas, Texas, USA, 295--310.
[51]
Dominik Reichl. 2018. KeePass (Windows) -- Password Manager. http://keepass.info/help/kb/pw_quality_est.html, as of today.
[52]
Dominik Reichl. 2018. KPScript (Windows) -- Scripting KeePass. http://keepass.info/help/v2_dev/scr_index.html, as of today.
[53]
Stuart Schechter, Cormac Herley, and Michael Mitzenmacher. 2010. Popularity Is Everything: A New Approach to Protecting Passwords from Statistical-Guessing Attacks. In USENIX Workshop on Hot Topics in Security (HotSec '10). USENIX, Washington, District of Columbia, USA.
[54]
Sinew Software Systems. 2016. Enpass Release Notes -- Use of the zxcvbn Strength Meter. https://www.enpass.io/release-notes/windowspc/, as of today.
[55]
Sinew Software Systems. 2018. Enpass (Windows) -- Password Manager. https://www.enpass.io, as of today.
[56]
Eugene H. Spafford. 1992. Observing Reusable Password Choices. In USENIX Security Symposium (SSYM '92). USENIX, Berkeley, California, USA, 299--312.
[57]
Richard Tilley. 2018. Blooming Password. https://www.bloomingpassword.fun, as of today.
[58]
Blase Ur. 2017. Source Code -- Data-Driven Password Meter. https://github.com/cupslab/password_meter, as of today.
[59]
Blase Ur, Felicia Alfieri, Maung Aung, Lujo Bauer, Nicolas Christin, Jessica Colnago, Lorrie Faith Cranor, Henry Dixon, Pardis Emami Naeini, Hana Habib, Noah Johnson, and William Melicher. 2017. Design and Evaluation of a Data-Driven Password Meter ACM Conference on Human Factors in Computing Systems (CHI '17). ACM, Denver, Colorado, USA, 3775--3786.
[60]
Blase Ur, Jonathan Bees, Sean M. Segreti, Lujo Bauer, Nicolas Christin, and Lorrie Faith Cranor. 2016. Do Users' Perceptions of Password Security Match Reality? ACM Conference on Human Factors in Computing Systems (CHI '16). ACM, Santa Clara, California, USA, 3748--3760.
[61]
Blase Ur, Patrick Gage Kelley, Saranga Komanduri, Joel Lee, Michael Maass, Michelle L. Mazurek, Timothy Passaro, Richard Shay, Timothy Vidas, Lujo Bauer, Nicolas Christin, and Lorrie Faith Cranor. 2012. How Does Your Password Measure Up? The Effect of Strength Meters on Password Creation USENIX Security Symposium (SSYM '12). USENIX, Bellevue, Washington, USA, 65--80.
[62]
Blase Ur, Sean M. Segreti, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Saranga Komanduri, Darya Kurilova, Michelle L. Mazurek, William Melicher, and Richard Shay. 2015. Measuring Real-World Accuracies and Biases in Modeling Password Guessability USENIX Security Symposium (SSYM '15). USENIX, Washington, D.C., USA, 463--481.
[63]
Ashlee Vance. 2010. If Your Password Is 123456, Just Make It HackMe. http://www.nytimes.com/2010/01/21/technology/21password.html, as of today.
[64]
Rafael Veras, Christopher Collins, and Julie Thorpe. 2014. On the Semantic Patterns of Passwords and their Security Impact Symposium on Network and Distributed System Security (NDSS '14). The Internet Society, San Diego, California, USA.
[65]
Ding Wang, Debiao He, Haibo Cheng, and Ping Wang. 2016. fuzzyPSM: A New Password Strength Meter Using Fuzzy Probabilistic Context-Free Grammars. In Conference on Dependable Systems and Networks (DSN '16). IEEE Computer Society, Toulouse, France, 595--606.
[66]
Ding Wang, Zijian Zhang, Ping Wang, Jeff Yan, and Xinyi Huang. 2016. Targeted Online Password Guessing: An Underestimated Threat ACM Conference on Computer and Communications Security (CCS '16). ACM, Vienna, Austria, 1242--1254.
[67]
Miranda Wei, Maximilian Golla, and Blase Ur. 2018. The Password Doesn't Fall Far: How Service Influences Password Choice Who Are You?! Adventures in Authentication Workshop (WAY '18). USENIX, Baltimore, Maryland, USA.
[68]
Matt Weir, Sudhir Aggarwal, Michael Collins, and Henry Stern. 2010. Testing Metrics for Password Creation Policies by Attacking Large Sets of Revealed Passwords. In ACM Conference on Computer and Communications Security (CCS '10). ACM, Chicago, Illinois, USA, 162--175.
[69]
Matt Weir, Sudhir Aggarwal, Breno de Medeiros, and Bill Glodek. 2009. Password Cracking Using Probabilistic Context-Free Grammars IEEE Symposium on Security and Privacy. IEEE Computer Society, Oakland, California, USA, 391--405.
[70]
Matt Weir, Sudhir Aggarwal, Breno De Medeiros, and Bill Glodek. 2009. Password Cracking Using Probabilistic Context-Free Grammars IEEE Symposium on Security and Privacy (SP '09). IEEE, Berkeley, CA, USA, 391--405.
[71]
Daniel Lowe Wheeler. 2016. zxcvbn: Low-Budget Password Strength Estimation. In USENIX Security Symposium (SSYM '16). USENIX, Austin, Texas, USA, 157--173.

Cited By

View all
  • (2024)The impact of exposed passwords on honeyword efficacyProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698932(559-576)Online publication date: 14-Aug-2024
  • (2024)The Effect of Domain Terms on Password SecurityACM Transactions on Privacy and Security10.1145/370335028:1(1-29)Online publication date: 4-Nov-2024
  • (2024)Priming through Persuasion: Towards Secure Password BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36373878:CSCW1(1-27)Online publication date: 26-Apr-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CCS '18: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
October 2018
2359 pages
ISBN:9781450356930
DOI:10.1145/3243734
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. password
  2. strength meter
  3. user authentication

Qualifiers

  • Research-article

Conference

CCS '18
Sponsor:

Acceptance Rates

CCS '18 Paper Acceptance Rate 134 of 809 submissions, 17%;
Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)205
  • Downloads (Last 6 weeks)21
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)The impact of exposed passwords on honeyword efficacyProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3698932(559-576)Online publication date: 14-Aug-2024
  • (2024)The Effect of Domain Terms on Password SecurityACM Transactions on Privacy and Security10.1145/370335028:1(1-29)Online publication date: 4-Nov-2024
  • (2024)Priming through Persuasion: Towards Secure Password BehaviorProceedings of the ACM on Human-Computer Interaction10.1145/36373878:CSCW1(1-27)Online publication date: 26-Apr-2024
  • (2024) GuessFuse : Hybrid Password Guessing With Multi-View IEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.337624619(4215-4230)Online publication date: 2024
  • (2024)Universal Neural-Cracking-Machines: Self-Configurable Password Models from Auxiliary Data2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00032(1365-1384)Online publication date: 19-May-2024
  • (2024)Contrasting and Synergizing CISOs' and Employees' Attitudes, Needs, and Resources for Security Using Personas2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00058(456-472)Online publication date: 8-Jul-2024
  • (2024)Increase Security by Analyzing Password Strength using Machine Learning2024 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON)10.1109/ECTIDAMTNCON60518.2024.10479995(32-37)Online publication date: 31-Jan-2024
  • (2024)A Study on Markov-Based Password Strength MetersIEEE Access10.1109/ACCESS.2024.340119512(69066-69075)Online publication date: 2024
  • (2024)Decoding developer password patterns: A comparative analysis of password extraction and selection practicesComputers & Security10.1016/j.cose.2024.103974145(103974)Online publication date: Oct-2024
  • (2024)PassTSL: Modeling Human-Created Passwords Through Two-Stage LearningInformation Security and Privacy10.1007/978-981-97-5101-3_22(404-423)Online publication date: 15-Jul-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media