Abstract
Passwords are the major part of authentication in current social networks. The state-of-the-art password guessing approaches, such as Markov model and probabilistic context-free grammars (PCFG) model, assign a probability value to each password by a statistic approach without any parameters. These methods require large datasets to accurately estimate probability due to the law of large number. The neural network, approximating target probability distribution through iteratively training its parameters, was used to model passwords by some researches. However, since the network architectures they used are simple and straightforward, there are many ways to improve it.
In this paper, we view password guessing as a language modeling task and introduce a deeper, more robust, and faster-converged model with several useful techniques to model passwords. This model shows great ability in modeling passwords while significantly outperforms state-of-the-art approaches. Inspired by the most advanced sequential model named Transformer, we use it to model passwords with bidirectional masked language model which is powerful but unlikely to provide normalized probability estimation. Then we distill Transformer model’s knowledge into our proposed model to further boost its performance. Comparing with the PCFG, Markov and previous neural network models, our models show remarkable improvement in both one-site tests and cross-site tests. Moreover, our models are robust to the password policy by controlling the entropy of output distribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
John the ripper (1996). https://www.openwall.com/john/
Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)
Castelluccia, C., Dürmuth, M., Perito, D.: Adaptive password-strength meters from Markov models. In: NDSS (2012)
Dell’Amico, M., Filippone, M.: Monte carlo strength evaluation: fast and reliable password checking. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 158–169. ACM (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dürmuth, M., Angelstorf, F., Castelluccia, C., Perito, D., Chaabane, A.: OMEN: faster password guessing using an ordered Markov enumerator. In: Piessens, F., Caballero, J., Bielova, N. (eds.) ESSoS 2015. LNCS, vol. 8978, pp. 119–132. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-15618-7_10
Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 1019–1027 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Hitaj, B., Gasti, P., Ateniese, G., Perez-Cruz, F.: PassGAN: a deep learning approach for password guessing. In: Deng, R.H., Gauthier-Umaña, V., Ochoa, M., Yung, M. (eds.) ACNS 2019. LNCS, vol. 11464, pp. 217–237. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21568-2_11
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1996)
Houshmand, S., Aggarwal, S., Flood, R.: Next gen PCFG password cracking. IEEE Trans. Inf. Forensics Secur. 10(8), 1776–1791 (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Hashcat (2009). https://hashcat.net/oclhashcat/
Kelley, P.G., et al.: Guess again (and again and again): measuring password strength by simulating password-cracking algorithms. In: 2012 IEEE Symposium on Security and Privacy (SP), pp. 523–537. IEEE (2012)
Krause, B., Kahembwe, E., Murray, I., Renals, S.: Dynamic evaluation of neural sequence models. arXiv preprint arXiv:1709.07432 (2017)
Li, Z., Han, W., Xu, W.: A large-scale empirical analysis of chinese web passwords. In: USENIX Security Symposium, pp. 559–574 (2014)
Liu, Y., et al.: GENPass: a general deep learning model for password guessing with PCFG rules and adversarial generation. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2018)
Ma, J., Yang, W., Luo, M., Li, N.: A study of probabilistic password models. In: 2014 IEEE Symposium on Security and Privacy (SP), pp. 689–704. IEEE (2014)
Melicher, W., et al.: Fast, lean, and accurate: modeling password guessability using neural networks. In: USENIX Security Symposium, pp. 175–191 (2016)
Merity, S., Keskar, N.S., Socher, R.: Regularizing and optimizing LSTM language models. arXiv preprint arXiv:1708.02182 (2017)
Myspace. https://www.myspace.com/
Narayanan, A., Shmatikov, V.: Fast dictionary attacks on passwords using time-space tradeoff. In: Proceedings of the 12th ACM Conference on Computer and Communications Security, pp. 364–372. ACM (2005)
phpBB. https://www.phpbb.com/
RockYou. https://www.rockyou.com/
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Tang, Z., Wang, D., Zhang, Z.: Recurrent neural network training with dark knowledge transfer. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5900–5904. IEEE (2016)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Weir, M., Aggarwal, S., De Medeiros, B., Glodek, B.: Password cracking using probabilistic context-free grammars. In: 2009 30th IEEE Symposium on Security and Privacy, pp. 391–405. IEEE (2009)
Xu, L., et al.: Password guessing based on LSTM recurrent neural networks. In: 2017 IEEE International Conference on Computational Science and Engineering (CSE) and Embedded and Ubiquitous Computing (EUC), vol. 1, pp. 785–788. IEEE (2017)
Yahoo!. https://www.yahoo.com/
Yang, Z., Dai, Z., Salakhutdinov, R., Cohen, W.W.: Breaking the softmax bottleneck: a high-rank RNN language model. arXiv preprint arXiv:1711.03953 (2017)
Acknowledgment
The authors are grateful to the anonymous reviewers for their constructive comments. This work was supported in part by the National Natural Science Foundation of China under Grant 61702399 and Grant 61772291 and Grant 61972215 in part by the Natural Science Foundation of Tianjin, China, under Grant 17JCZDJC30500.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H., Chen, M., Yan, S., Jia, C., Li, Z. (2019). Password Guessing via Neural Language Modeling. In: Chen, X., Huang, X., Zhang, J. (eds) Machine Learning for Cyber Security. ML4CS 2019. Lecture Notes in Computer Science(), vol 11806. Springer, Cham. https://doi.org/10.1007/978-3-030-30619-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-30619-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30618-2
Online ISBN: 978-3-030-30619-9
eBook Packages: Computer ScienceComputer Science (R0)