Skip to main content
SpringerLink
Log in

Handwritten CAPTCHA recognizer: a text CAPTCHA breaking method based on style transfer network

  • 1184: Security and Privacy for Intelligent Multimedia Processing in the Era of Big Data
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The CAPTCHA technology can be used to ensure big multimedia data security, which includes CAPTCHA design and CAPTCHA recognition. For the existing methods are difficult to achieve high breaking accuracy for complex handwritten text CAPTCHA, a handwritten CAPTCHA recognizer is proposed, which is a text CAPTCHA breaking method based on style transfer network. Firstly, different from the traditional viewpoints that font structure and font style of characters are inseparable in this field, a new idea of separating font structure and font style of characters is proposed, and it is pointed out that character recognition mainly depends on font structure rather than font style. Secondly, based on this idea, a style transfer network for text CAPTCHA is constructed to convert complex and variable handwritten CAPTCHA into easy-to-recognize printed CAPTCHA. Finally, based on deep convolutional neural network, a text CAPTCHA recognition network is constructed to identify the converted printed CAPTCHAs. With CAPTCHAs from three real websites: eBay, Google and reCAPTCHA, experimental results show that the recognizer has higher breaking accuracy for handwritten CAPTCHA compared with the methods proposed in NDSS’16, CCS’18 and “Science” in 2017.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Alec R, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint. https://arxiv.org/abs/1511.06434

  2. Brock A, Donahue J, Simonyan K (2019) Large scale GAN training for high fidelity natural image synthesis, In Proceedings of the International Conference on Learning Representations (ICLR). https://openreview.net/pdf?id=B1xsqj09Fm

  3. Bursztein E, Martin M, Mitchell JC (2011) Text-based CAPTCHA strengths and weaknesses, In Proceedings of the ACM Conference on Computer and Communications Security (CCS) 125–137

  4. Chang J, Gu Y, Zhang Y (2017) Chinese typography transfer, arXiv preprint. https://arxiv.org/abs/1707.04904

  5. Frid-Adar M, Diamant I et al (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321:321–331

    Article  Google Scholar 

  6. Fiot JB, Paucher R (2009) The Captchacker project, Ecole Centrale Paris

  7. Gao H, Wang W, Fan Y (2012) Divide and conquer: An efficient attack on Yahoo! CAPTCHA, In Proceedings of the IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) 9–16

  8. Gao H, Yan J et al (2016) A simple generic attack on text CAPTCHAs, In Proceedings of the Network and Distributed System Security Symposium (NDSS) 1–14

    Google Scholar 

  9. George D, Lehrach W et al (2017) A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs. Science 356(6368):eaag2612

  10. Goodfellow IJ, Bulatov Y et al (2014) Multi-digit number recognition from street view imagery using deep convolutional neural networks, In Proceedings of the International Conference on Learning Representations (ICLR). https://www.researchgate.net/publication/259399973

  11. Goodfellow I, Papernot N, Huang S (2017) Attacking machine learning with adversarial examples, OpenAI. https://blog.openai.com/adversarial-example-research/

  12. Guang Z, Wan Z et al (2020) BlockMaze: an efficient privacy-preserving account-model blockchain based on zk-SNARKs. IEEE Transactions on Dependable and Secure Computing. https://doi.org/10.1109/TDSC.2020.3025129

    Article  Google Scholar 

  13. Han D, Pan N, Li K (2020) A traceable and revocable ciphertext-policy attribute-based encryption scheme based on privacy protection. IEEE Transactions on Dependable and Secure Computing. https://doi.org/10.1109/TDSC.2020.2977646

    Article  Google Scholar 

  14. Huang H, Ma X et al (2021) Unlearnable examples: Making personal data unexploitable. In Proceedings of International Conference on Learning Representations (ICLR) arXiv preprint. https://arxiv.org/abs/2101.04898

  15. Isola P, Zhu J, Zhou T et al (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2017.632

    Article  Google Scholar 

  16. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 4401-4410

  17. Liu T, Wang Y et al (2020) Privacy protection based on stream cipher for spatiotemporal data in IoT. IEEE Internet of Things Journal 7(9):7928–7940

    Article  Google Scholar 

  18. Matan O, Burges CJC et al (1991) Multi-digit recognition using a space displacement neural network, In Proceedings of Advances in Neural Information Processing Systems (NIPS) 488–495

  19. Moy G, Jones N et al (2004) Distortion estimation techniques in solving visual CAPTCHAs, In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 1123–1128

  20. Pu Y, Xiang T et al (2020) An efficient blockchain-based privacy preserving scheme for vehicular social networks. Inform Sci 540:308–324

    Article  MathSciNet  Google Scholar 

  21. Starostenko O, Cruz-Perez C et al (2015) Breaking text-based CAPTCHAs with variable word and character orientation. Patt Recognit 48(4):1097–1108

    Article  Google Scholar 

  22. Von Ahn L, Blum M et al (2003) CAPTCHA: Using hard AI problems for security, In Proceedings of International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT) LNCS 2656:294–311

  23. Yan J, El Ahmad AS (2007) Breaking visual CAPTCHAs with native pattern recognition algorithms, In Proceedings of the Annual Computer Security Applications Conference (ACSAC) 279–291

  24. Yang Y, Deng RH et al (2020) Privacy-preserving medical treatment system through nondeterministic finite automata. IEEE Transactions on Cloud Computing. https://doi.org/10.1109/TCC.2020.2999940

    Article  Google Scholar 

  25. Ye G, Tang Z et al (2018) Yet another text CAPTCHA solver: A generative adversarial network based approach, In Proceedings of the ACM Conference on Computer and Communications Security (CCS) 332–348

  26. Yosinski J, Clune J et al (2014) How transferable are features in deep neural networks? In Proceedings of Advances in Neural Information Processing Systems (NIPS) 3320–3328

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. U1804263, 62172435) and the Zhongyuan Science and Technology Innovation Leading Talent Project of China (Grant No. 214200510019).

Author information

Authors and Affiliations

  1. Nanjing Sport Institute, Nanjing, China

    Jun Chen

  2. State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou, China

    Xiangyang Luo & Liyan Zhu

  3. Zhengzhou University of Light Industry, Zhengzhou, China

    Qikun Zhang & Yong Gan

Authors
  1. Jun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangyang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liyan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qikun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangyang Luo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Luo, X., Zhu, L. et al. Handwritten CAPTCHA recognizer: a text CAPTCHA breaking method based on style transfer network. Multimed Tools Appl 82, 13025–13043 (2023). https://doi.org/10.1007/s11042-021-11485-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11485-9

Keywords

Search

Navigation