Skip to main content
SpringerLink
Log in

End-to-End Multi-line License Plate Recognition with Cascaded Perception

  • Conference paper
  • First Online:
Document Analysis and Recognition - ICDAR 2023 (ICDAR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14191))

Included in the following conference series:

Abstract

Due to the irregular layout, multi-line license plates are challenging to recognize, and previous methods cannot recognize them effectively and efficiently. In this work, we propose an end-to-end multi-line license plate recognition network, which cascades global type perception and parallel character perception to enhance recognition performance and inference speed. Specifically, we first utilize self-information mining to extract global features to perceive plate type and character layout, improving recognition performance. Then, we use the reading order to attend plate characters parallelly, strengthening inference speed. Finally, we propose extracting recognition features from shallow layers of the backbone to construct an end-to-end detection and recognition network. This way, it can reduce error accumulation and retain more plate information, such as character stroke and layout, to enhance recognition. Experiments on three datasets prove our method can achieve state-of-the-art recognition performance, and cross-dataset experiments on two datasets verify the generality of our method. Moreover, our method can achieve a breakneck inference speed of 104 FPS with a small backbone while outperforming most comparative methods in recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. MobileNetV1 SSD (2023). https://github.com/qfgaohao/pytorch-ssd

  2. OpenALPR Cloud API (2023). https://www.openalpr.com/carcheck-api.html

  3. Baek, J., Kim, G., Lee, J., et al.: What is wrong with scene text recognition model comparisons? dataset and model analysis. In: ICCV, pp. 4714–4722 (2019)

    Google Scholar 

  4. Cao, Y., Fu, H., Ma, H.: An end-to-end neural network for multi-line license plate recognition. In: ICPR, pp. 3698–3703 (2018)

    Google Scholar 

  5. Chen, S., Tian, S., Ma, J., et al.: End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining. Neurocomputing 446, 1–10 (2021)

    Article  Google Scholar 

  6. Chen, S., Yang, C., Ma, J., et al.: Simultaneous end-to-end vehicle and license plate detection with multi-branch attention neural network. IEEE Trans. Intell. Transp. Syst. 21(9), 3686–3695 (2020)

    Article  Google Scholar 

  7. Dong, M., He, D., Luo, C., et al.: A cnn-based approach for automatic license plate recognition in the wild. In: BMVC, pp. 1–12 (2017)

    Google Scholar 

  8. Duan, S., Hu, W., Li, R., et al.: Attention enhanced convnet-rnn for Chinese vehicle license plate recognition. In: PRCV, pp. 417–428 (2018)

    Google Scholar 

  9. Girshick, R.: Fast R-CNN. In: ICCV, pp. 1440–1448 (2015)

    Google Scholar 

  10. Howard, A., Zhu, M., Chen, B., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications (2017). arXiv:1704.04861

  11. Jain, V., Sasindran, Z., Rajagopal, A., et al.: Deep automatic license plate recognition system. In: ICVGIP, pp. 1–8 (2016)

    Google Scholar 

  12. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: ICLR, pp. 1–15 (2015)

    Google Scholar 

  13. Laroca, R., Cardoso, E., Lucio, D.: On the cross-dataset generalization in license plate recognition. In: VISIGRAPP, pp. 166–178 (2022)

    Google Scholar 

  14. Laroca, R., Severo, E., Zanlorensi, L., et al.: A robust real-time automatic license plate recognition based on the YOLO detector. In: IJCNN, pp. 1–10 (2018)

    Google Scholar 

  15. Laroca, R., Zanlorensi, L., Gonçalves, G., et al.: An efficient and layout-independent automatic license plate recognition system based on the YOLO detector. IET Intell. Transp. Syst. 15(4), 483–503 (2021)

    Article  Google Scholar 

  16. Li, H., Wang, P., Shen, C.: Towards end-to-end car license plates detection and recognition with deep neural networks. IEEE Trans. Intell. Transp. Syst. 20(3), 1126–1136 (2019)

    Article  Google Scholar 

  17. Li, H., Wang, P., Shen, C., et al.: Show, attend and read: a simple and strong baseline for irregular text recognition. In: AAAI, pp. 8610–8617 (2019)

    Google Scholar 

  18. Lin, T., Maji, S.: Visualizing and understanding deep texture representations. In: CVPR, pp. 2791–2799 (2016)

    Google Scholar 

  19. Liu, Q., Chen, S., Li, Z., et al.: Fast recognition for multidirectional and multi-type license plates with 2D spatial attention. In: ICDAR, pp. 125–139 (2021)

    Google Scholar 

  20. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  21. Luo, C., Jin, L., Sun, Z.: MORAN: a multi-object rectified attention network for scene text recognition. Pattern Recogn. 90, 109–118 (2019)

    Article  Google Scholar 

  22. Masood, S.Z., Shu, G., Dehghan, A., et al.: License plate detection and recognition using deeply learned convolutional neural networks (2017). arXiv:1703.07330

  23. Qiao, L., Chen, Y., Cheng, Z., et al.: MANGO: a mask attention guided one-stage scene text spotter. In: AAAI, pp. 2467–2476 (2021)

    Google Scholar 

  24. Qin, S., Liu, S.: Towards end-to-end car license plate location and recognition in unconstrained scenarios. Neural Comput. Appl. 34(24), 21551–21566 (2022)

    Article  Google Scholar 

  25. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2017)

    Article  Google Scholar 

  26. Silva, S., Jung, C.: License plate detection and recognition in unconstrained scenarios. In: ECCV, pp. 593–609 (2018)

    Google Scholar 

  27. Silva, S., Jung, C.: Real-time license plate detection and recognition using deep convolutional neural networks. J. Vis. Commun. Image Represent. 71, 102773 (2020)

    Article  Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR, pp. 1–14 (2015)

    Google Scholar 

  29. Spanhel, J., Sochor, J., Juránek, R., et al.: Holistic recognition of low quality license plates by CNN using track annotated data. In: AVSS, pp. 1–6 (2017)

    Google Scholar 

  30. Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  31. Wang, T., Zhu, Y., Jin, L., et al.: Decoupled attention network for text recognition. In: AAAI, pp. 12216–12224 (2020)

    Google Scholar 

  32. Wang, Y., Bian, Z., Zhou, Y., et al.: Rethinking and designing a high-performing automatic license plate recognition approach. IEEE Trans. Intell. Transp. Syst. 23(7), 8868–8880 (2022)

    Article  Google Scholar 

  33. Xu, H., Zhou, X., Li, Z., et al.: EILPR: toward end-to-end irregular license plate recognition based on automatic perspective alignment. IEEE Trans. Intell. Transp. Syst. 23(3), 2586–2595 (2022)

    Article  Google Scholar 

  34. Xu, Z., Yang, W., Meng, A., et al.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. In: ECCV, pp. 261–277 (2018)

    Google Scholar 

  35. Yang, Y., Xi, W., Zhu, C., et al.: Homonet: unified license plate detection and recognition in complex scenes. In: CollaborateCom, pp. 268–282 (2020)

    Google Scholar 

  36. Yu, D., Li, X., Zhang, C., et al.: Towards accurate scene text recognition with semantic reasoning networks. In: CVPR, pp. 12110–12119 (2020)

    Google Scholar 

  37. Yuan, Y., Zou, W., Zhao, Y., et al.: A robust and efficient approach to license plate detection. IEEE Trans. Image Process. 26(3), 1102–1114 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhang, L., Wang, P., Li, H., et al.: A robust attentional framework for license plate recognition in the wild. IEEE Trans. Intell. Transp. Syst. 22(11), 6967–6976 (2021)

    Article  Google Scholar 

  39. Zheng, Z., Wang, P., Liu, W., et al.: Distance-iou loss: faster and better learning for bounding box regression. In: AAAI, pp. 12993–13000 (2020)

    Google Scholar 

  40. Zherzdev, S., Gruzdev, A.: Lprnet: license plate recognition via deep neural networks (2018). arXiv:1806.10447

  41. Zhuang, J., Hou, S., Wang, Z., et al.: Towards human-level license plate recognition. In: ECCV, pp. 314–329 (2018)

    Google Scholar 

  42. Zou, Y., Zhang, Y., Yan, J., et al.: License plate detection and recognition based on yolov3 and ILPRNET. Signal Image Video Process. 16(2), 473–480 (2022)

    Article  Google Scholar 

Download references

Acknowledgement

The research is supported by National Key Research and Development Program of China (2020AAA0109700), National Science Fund for Distinguished Young Scholars (62125601), and National Natural Science Foundation of China (62076024, 62006018, U22B2055).

Author information

Authors and Affiliations

  1. University of Science and Technology Beijing, Beijing, China

    Song-Lu Chen, Qi Liu & Xu-Cheng Yin

  2. EEasy Technology Company Ltd., Zhuhai, China

    Feng Chen

Authors
  1. Song-Lu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xu-Cheng Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu-Cheng Yin .

Editor information

Editors and Affiliations

  1. TU Dortmund University, Dortmund, Germany

    Gernot A. Fink

  2. Adobe, College Park, MN, USA

    Rajiv Jain

  3. Osaka Metropolitan University, Osaka, Japan

    Koichi Kise

  4. Rochester Institute of Technology, Rochester, NY, USA

    Richard Zanibbi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, SL., Liu, Q., Chen, F., Yin, XC. (2023). End-to-End Multi-line License Plate Recognition with Cascaded Perception. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14191. Springer, Cham. https://doi.org/10.1007/978-3-031-41734-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-41734-4_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-41733-7

  • Online ISBN: 978-3-031-41734-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation