Abstract
The paper focuses on the comparative analysis of deep learning algorithms and traditional probabilistic models on strings of short lengths (typically, passwords). The password is one of the dominant methods used in user authentication. Compared to the traditional brute-force attack and dictionary attack, password guessing models use the leaked password datasets to generate password guesses, expecting to cover as many accounts as possible while minimizing the number of guesses. In this paper, we analyze the password pattern of leaked datasets and further present a comparative study on two dominant probabilistic models (i.e., Markov-based model and Probabilistic Context-Free Grammars (PCFG) based model) and the PassGAN model (which is a representative deep-learning-based method).
We use Laplace smoothing for the Markov model and introduce particular semantic patterns to the PCFG model. Our output shows that the Markov-based models can cover the vast majority of the passwords in the test set and PassGAN demonstrates surprisingly the worst effect. Nevertheless, considering the threat that an attacker may adjust the training set, the PCFG model is better than the Markov model. Using Passcode with high-frequency passwords can increase the coverage while reducing the number of guesses. Brute-force attack can also work better when used in conjunction with probabilistic models. For the same billion guesses, brute-force attack can be used to crack pure digital passwords of 4 to 8 lengths, and original-PCFG and modified-PCFG could increase by 11.16% and 8.69%, respectively.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The interchange of the 17173 and 7k7k datasets is feasible. Whereas, the choice of the two datasets is not arbitrary, but stems from their substantive comparability (same language, similar data sizes, same web services (see Table 1)) so that the resulting comparison is meaningful.
- 2.
When looking into password guessing attacks, one may observe that it is pretty hard to obtain even a small leap for the coverage (especially with the increasing of the number of guesses). For example, one can exploit modified-PCFG model to produce 1 billion passwords which are then used to crack the 17173 training set. Next, take the difference set (i.e., all the passwords not covered by the collection of the 1 billion passwords) as new training set and generate 200 million more passwords. Now the coverage gain is only 0.55% (contrary to expectations). This kind of striking contrast (in experiments) may not be thoroughly experienced from comparison exhibition (in a figure). Whereas, as can be seen from Fig. 2, it is very clear that the Markov-based model reports significantly better performance than the PCFG-based model when the number of guesses reaches 1 billion.
The models themselves are suitable for different datasets/languages (which will affect the resulting coverage and lead to different outputs). Generally, a cut-and-dried dictionary would be exploited for the dataset of a specific user language to capture the semantic pattern of the user group (for PCFG-based models). However, this does not mean that the models are not applicable to other languages.
- 3.
This experiment package manifests that small datasets may convey incomplete information, and this incompleteness could expose inherent defects especially when the number of guesses is large. According to the rationale of these models, even with 1% of the training set, the models could generate 1 billion password guesses (used to conduct comparative experiments). Yet what’s more concerning is the coverage in the context of password guessing. For the probabilistic model, the password used for the N-th guessing is of the N-th largest probability produced by the model.
References
Zhu, B., Yan, J., Bao, G., Yang, M., Xu, N.: Captcha as graphical passwords- a new security primitive based on hard AI problems. IEEE Trans. Inf. Forensics Secur. 9, 234–240 (2014)
van Herley, C., Oorschot, P.: A research agenda acknowledging the persistence of passwords. IEEE Secur. Priv. 10, 28–36 (2012)
Burr, W.E., et al.: NIST SP800-63-2: Electronic authentication guideline. National Institute of Standards and Technology, Reston, VA, Technical report, Special Publication (NIST SP) - 800–63-1 (2011)
Matt, W., Sudhir, A., et al.: Testing metrics for password creation policies by attacking large sets of revealed passwords. In: ACM CCS 2010, pp. 162–175 (2010)
Matt, W., Sudhir, A., et al.: Password cracking using probabilistic context-free grammars. In: IEEE Symposium on Security and Privacy, pp. 391–405 (2009)
Houshmand, S., Aggarwal, S., Flood, R.: Next gen PCFG password cracking. IEEE Trans. Inf. Forensics Secur. 10, 1776–1791 (2015)
Wang, D., Cheng, H., et al.: Understanding Passwords of Chinese Users: Characteristics, Security and Implications, July 2014. https://www.researchgate.net/profile/Ding_Wang12/publication/269101022_Understanding_Passwords_of_Chinese_Users_Characteristics_Security_and_Implications/links/5544e2700cf23ff7168696a8.pdf
Veras, R., Collins, C., Thorpe, J.: On the semantic patterns of passwords and their security impact. In: Network and Distributed System Security Symposium (2014)
Arvind, N., Vitaly, S.: Fast dictionary attacks on passwords using time-space tradeoff. In: ACM CCS 2005, pp. 364–372 (2005)
Ma, J., Yang, W., Luo, M., Li, N.: A study of probabilistic password models. In: Proceedings of IEEE Symposium on Security and Privacy, pp. 689–704 (2014)
Hitaj, B., Gasti, P., Ateniese, G., Pérez-Cruz, F.: PassGAN: a deep learning approach for password guessing. CoRR abs/1709.00440 (2017)
Xu, L., Ge, C., et al.: Password guessing based on LSTM recurrent neural networks. In: IEEE International Conference on Computational Science and Engineering, pp. 785–788 (2017)
Li, Y., Wang, H., Sun, K.: A study of personal information in human-chosen passwords and its security implications. In: IEEE INFOCOM, pp. 1-9 (2016)
Wang, D., Zhang, Z., Wang, P., Yan, J., Huang, X.: An underestimated threat. In: ACM CCS, Targeted Online Password Guessing (2016)
ChineseTone. https://github.com/letiantian/ChineseTone
English words. http://www.mieliestronk.com/wordlist.html
Gulrajani, I., Ahmed, F., et al.: Improved Training of Wasserstein GANs. CoRR abs/1704.00028 (2017). http://arxiv.org/abs/1704.00028
Acknowledgement
The paper is supported by the National Natural Science Foundation of China (Grant Nos. 61572192, 61971192) and the National Cryptography Development Fund (Grant No. MMJJ20180106).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Linghu, Y., Li, X., Zhang, Z. (2020). Deep Learning vs. Traditional Probabilistic Models: Case Study on Short Inputs for Password Guessing. In: Wen, S., Zomaya, A., Yang, L. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11944. Springer, Cham. https://doi.org/10.1007/978-3-030-38991-8_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-38991-8_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38990-1
Online ISBN: 978-3-030-38991-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)