Skip to main content
SpringerLink
Log in

License plate detection and recognition based on YOLOv3 and ILPRNET

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper is concerned with the detection and recognition of Chinese license plates in complex backgrounds. Most applications are currently focused on good conditions. In complex natural scenes such as CCPD-DB, CCPD-FN, CCPD-Rotate, CCPD-Tile, CCPD-Weather, and CCPD-Challenge from the Chinese City Parking Dataset (CCPD), inaccurate localization and poor character recognition accuracy issues appear towards existing license plates. Therefore, this paper proposes a two-stage license plate recognition algorithm based on YOLOv3 and Improved License Plate Recognition Net (ILPRNET). In the first stage, YOLOv3 is adopted to detect the position of the license plate and then extract the license plate. In the second stage, the ILPRNET license plate recognition network is used to perform localization of license plate characters and the 2D attentional-based license plate recognizer with an CNN encoder is capable of recognizing license plates accurately. The test results indicate that our proposed algorithm performs well in a variety of complex scenarios. Especially in sub-datasets like CCPD-Base, CCPD-DB, CCPD-FN, CCPD-Weather, and CCPD-Challenge, the recognition accuracy achieved 99.2%, 98.1%, 98.5%, 97.8%, and 86.2%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. Preprint arXiv:2004.10934 (2020)

  2. Bulan, O., Kozitsky, V., Ramesh, P., Shreve, M.: Segmentation-and annotation-free license plate recognition with deep localization and failure identification. IEEE Trans. Intell. Transp. Syst. 18(9), 2351–2363 (2017)

    Article  Google Scholar 

  3. Cheng, Z., Bai, F., Xu, Y., Zheng, G., Pu, S., Zhou, S.: Focusing attention: towards accurate text recognition in natural images. In: IEEE International Conference on Computer Vision, pp. 5076–5084 (2017)

  4. Girshick, R.: Fast r-cnn. In: IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  5. Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. J. Electron. Imaging 25(5), 053034 (2016)

    Article  Google Scholar 

  6. Hsu, G.S., Chen, J.C., Chung, Y.Z.: Application-oriented license plate recognition. IEEE Trans. Veh. Technol. 62(2), 552–561 (2012)

    Article  Google Scholar 

  7. Lee, Y., Song, T., Ku, B., Jeon, S., Han, D.K., Ko, H.: License plate detection using local structure patterns. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 574–579 (2010)

  8. Li, H., Shen, C.: Reading car license plates using deep convolutional neural networks and lstms. Preprint arXiv:1601.05610 (2016)

  9. Li, H., Wang, P., Shen, C.: Toward end-to-end car license plate detection and recognition with deep neural networks. IEEE Trans. Intell. Transp. Syst. 20(3), 1126–1136 (2018)

    Article  Google Scholar 

  10. Li, H., Wang, P., Shen, C., Zhang, G.: Show, attend and read: a simple and strong baseline for irregular text recognition. AAAI Conf. Artif. Intell. 33, 8610–8617 (2019)

    Google Scholar 

  11. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37 (2016)

  12. Luo, C., Jin, L., Sun, Z.: A multi-object rectified attention network for scene text recognition. Preprint arXiv:1901.03003 (2019)

  13. Masood, S.Z., Shu, G., Dehghan, A., Ortiz, E.G.: License plate detection and recognition using deeply learned convolutional neural networks. Preprint arXiv:1703.07330 (2017)

  14. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  15. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

  16. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. Preprint arXiv:1804.02767 (2018)

  17. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. Preprint arXiv:1506.01497 (2015)

  18. Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., Bai, X.: Aster: an attentional scene text recognizer with flexible rectification. IEEE Trans. Pattern Anal. Mach. Intell. 41(9), 2035–2048 (2018)

  19. Silva, S.M., Jung, C.R.: License plate detection and recognition in unconstrained scenarios. In: European Conference on Computer Vision (ECCV), pp. 580–596 (2018)

  20. Wang, R., Sang, N., Huang, R., Wang, Y.: License plate detection using gradient information and cascade detectors. Optik 125(1), 186–190 (2014)

    Article  Google Scholar 

  21. Wang, S.Z., Lee, H.J.: A cascade framework for a real-time statistical plate recognition system. IEEE Trans. Inf. For. Secur. 2(2), 267–282 (2007)

    Article  Google Scholar 

  22. Wang, T., Zhu, Y., Jin, L., Luo, C., Chen, X., Wu, Y., Wang, Q., Cai, M.: Decoupled attention network for text recognition. AAAI Conf. Artif. Intel. 34, 12216–12224 (2020)

    Google Scholar 

  23. Wu, C., Xu, S., Song, G., Zhang, S.: How many labeled license plates are needed? In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 334–346 (2018)

  24. Xu, Z., Yang, W., Meng, A., Lu, N., Huang, H., Ying, C., Huang, L.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 255–271 (2018)

  25. Yao, Z., Yi, W.: License plate detection based on multistage information fusion. Inf. Fusion 18, 78–85 (2014)

    Article  Google Scholar 

  26. Yousef, M., Hussain, K.F., Mohammed, U.S.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recogn. 108, 107482 (2020)

    Article  Google Scholar 

  27. Yu, S., Li, B., Zhang, Q., Liu, C., Meng, M.Q.H.: A novel license plate location method based on wavelet transform and emd analysis. Pattern Recogn. 48(1), 114–125 (2015)

    Article  Google Scholar 

  28. Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)

    Article  Google Scholar 

  29. Zhang, L., Wang, P., Li, H., Li, Z., Shen, C., Zhang, Y.: A robust attentional framework for license plate recognition in the wild. IEEE Trans. Intell. Transp. Syst. (2020)

  30. Zherzdev, S., Gruzdev, A.: Lprnet: license plate recognition via deep neural networks. Preprint arXiv:1806.10447 (2018)

  31. Zou, Y., Zhang, Y., Yan, J., Jiang, X., Huang, T., Fan, H., Cui, Z.: A robust license plate recognition model based on bi-lstm. IEEE Access 8, 211630–211641 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by 2017 Zhuhai introduces innovation and entrepreneurship team (ZH01110405170027PWC, No.ZH0405-1900-01PWC) and Cloud Service Platform and Applications based on “ZHUHAI No.1” Constellation & Remote Sensing Big Data.(ZDXK[2018]007), Key Supported Disciplines of Guizhou Province-Computer Application Technology (No. QianXueWeiHeZi ZDXK [2016]20), and the work was also supported by National Natural Science Foundation of China (61462013, 61661010)

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Medical Image Analysis and Precise Diagnosis of Guizhou Province, College of Computer Science and Technology, Guizhou University, Guiyang, 550025, China

    Yongjie Zou & Yongjun Zhang

  2. Zhuhai Orbita Aerospace Science and Techology Co, Ltd, Oribita Tech Park, 1 BaiSha Road, TangJia DongAn, ZhuHai, 519000, China

    Jun Yan, Xiaoxu Jiang, Tengjie Huang & Haisheng Fan

  3. Big Data Science and Intelligent Engineering Research Institute, Guizhou Education University, Guiyang, 550018, China

    Zhongwei Cui

Authors
  1. Yongjie Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongjun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoxu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tengjie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haisheng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhongwei Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yongjun Zhang or Jun Yan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, Y., Zhang, Y., Yan, J. et al. License plate detection and recognition based on YOLOv3 and ILPRNET. SIViP 16, 473–480 (2022). https://doi.org/10.1007/s11760-021-01981-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01981-8

Keywords

Search

Navigation