Skip to main content
SpringerLink
Log in

Improving Small License Plate Detection with Bidirectional Vehicle-Plate Relation

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14555))

Included in the following conference series:

  • 282 Accesses

Abstract

License plate detection is a critical component of license plate recognition systems. A challenge in this domain is detecting small license plates captured at a considerable distance. Previous researchers have proved that pre-detecting the vehicle can enhance small license plate detection. However, this approach only utilizes the one-way relation that the presence of a vehicle can enhance license plate detection, potentially resulting in error accumulation if the vehicle fails to be detected. To address this issue, we propose a unified network that can simultaneously detect the vehicle and the license plate while establishing bidirectional relationships between them. The proposed network can utilize the vehicle to enhance small license plate detection and reduce error accumulation when the vehicle fails to be detected. Extensive experiments on the SSIG-SegPlate, AOLP, and CRPD datasets prove our method achieves state-of-the-art detection performance, achieving an average detection AP\(_{0.5}\) of 99.5% on these three datasets, especially for small license plates. When incorporating a license plate recognizer that relies on character detection, we can achieve an average recognition accuracy of 95.9%, surpassing all comparative methods. Moreover, we have manually annotated the vehicles within the CRPD dataset and have made these annotations publicly available at https://github.com/kiki00007/CRPDV.

S. Dai and S.-L. Chen—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Batra, P., Hussain, I., Ahad, M.A., Casalino, G., Alam, M.A., Khalique, A., Hassan, S.I.: A novel memory and time-efficient alpr system based on yolov5. Sensors 22(14), 5283 (2022)

    Article  Google Scholar 

  2. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)

  3. Borisyuk, F., Gordo, A., Sivakumar, V.: Rosetta: large scale system for text detection and recognition in images. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 71–79 (2018)

    Google Scholar 

  4. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13

    Chapter  Google Scholar 

  5. Chen, S.L., Liu, Q., Ma, J.W., Yang, C.: Scale-invariant multidirectional license plate detection with the network combining indirect and direct branches. Sensors 21(4), 1074 (2021)

    Article  Google Scholar 

  6. Chen, S.L., et al.: End-to-end trainable network for degraded license plate detection via vehicle-plate relation mining. Neurocomputing 446, 1–10 (2021)

    Article  Google Scholar 

  7. Chen, S.L., Yang, C., Ma, J.W., Chen, F., Yin, X.C.: Simultaneous end-to-end vehicle and license plate detection with multi-branch attention neural network. IEEE Trans. Intell. Transp. Syst. 21(9), 3686–3695 (2019)

    Article  Google Scholar 

  8. Glenn Jocher, Alex Stoken, J.B.: yolov5. https://github.com/ultralytics/yolov5. (2020)

  9. Gonçalves, G.R., da Silva, S.P.G., Menotti, D., Schwartz, W.R.: Benchmark for license plate character segmentation. J. Electron. Imaging 25(5), 053034–053034 (2016)

    Article  Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  11. Hsu, G.S., Chen, J.C., Chung, Y.Z.: Application-oriented license plate recognition. IEEE T-VT 62(2), 552–561 (2012)

    Google Scholar 

  12. Kim, S., Jeon, H., Koo, H.: Deep-learning-based license plate detection method using vehicle region extraction. Electron. Lett. 53(15), 1034–1036 (2017)

    Article  Google Scholar 

  13. Kuhn, H.W.: The Hungarian method for the assignment problem. Naval Res. Logistics Quarterly 2(1–2), 83–97 (1955)

    Article  MathSciNet  Google Scholar 

  14. Laroca, R., et al.: A robust real-time automatic license plate recognition based on the yolo detector. In: 2018 International Joint Conference on Neural Networks (ijcnn), pp. 1–10. IEEE (2018)

    Google Scholar 

  15. Laroca, R., Zanlorensi, L.A., Gonçalves, G.R., Todt, E., Schwartz, W.R., Menotti, D.: An efficient and layout-independent automatic license plate recognition system based on the yolo detector. IET Intel. Transport Syst. 15(4), 483–503 (2021)

    Article  Google Scholar 

  16. Li, H., Shen, C.: Reading car license plates using deep convolutional neural networks and lstms. arXiv preprint arXiv:1601.05610 (2016)

  17. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

    Google Scholar 

  18. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  19. Lin, T.Y., et al.: Microsoft coco: Common objects in context (2014)

    Google Scholar 

  20. Liu, W., Chen, C., Wong, K.Y.K., Su, Z., Han, J.: Star-net: a spatial attention residue network for scene text recognition. In: BMVC, vol. 2, p. 7 (2016)

    Google Scholar 

  21. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  22. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)

    Google Scholar 

  23. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S.: Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 658–666 (2019)

    Google Scholar 

  24. Selmi, Z., Halima, M.B., Alimi, A.M.: Deep learning system for automatic license plate detection and recognition. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1132–1138. IEEE (2017)

    Google Scholar 

  25. Selmi, Z., Halima, M.B., Pal, U., Alimi, M.A.: Delp-dar system for license plate detection and recognition. Pattern Recogn. Lett. 129, 213–223 (2020)

    Article  Google Scholar 

  26. Tian, Z., Shen, C., Chen, H., He, T.: Fcos: fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627–9636 (2019)

    Google Scholar 

  27. Vaswani, A., et al.: Attention is all you need. Advances in neural information processing systems 30 (2017)

    Google Scholar 

  28. Xu, Z., Yang, W., Meng, A., Lu, N., Huang, H., Ying, C., Huang, L.: Towards end-to-end license plate detection and recognition: a large dataset and baseline. In: Proceedings of the European conference on computer vision (ECCV), pp. 255–271 (2018)

    Google Scholar 

  29. Zhang, H., Yao, Q., Yang, M., Xu, Y., Bai, X.: Autostr: efficient backbone search for scene text recognition. In: Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV 16, pp. 751–767. Springer (2020)

    Google Scholar 

  30. Zhang, S., Chi, C., Yao, Y., Lei, Z., Li, S.Z.: Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9759–9768 (2020)

    Google Scholar 

  31. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)

  32. Zou, Y., Zhang, Y., Yan, J., Jiang, X., Huang, T., Fan, H., Cui, Z.: License plate detection and recognition based on yolov3 and ilprnet. SIViP 16(2), 473–480 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partly supported by the National Key Research and Development Program of China under Grant 2020AAA0109700 and partly by the National Natural Science Foundation of China under Grant 62076024 and Grant 62006018.

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China

    Songkang Dai, Song-Lu Chen, Qi Liu, Chao Zhu & Xu-Cheng Yin

  2. Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China

    Yan Liu

  3. EEasy Technology Company Ltd., Zhuhai, China

    Feng Chen

Authors
  1. Songkang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song-Lu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Feng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xu-Cheng Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Zhu .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  2. Delft University of Technology, Delft, The Netherlands

    Alan Hanjalic

  3. Delft University of Technology, Delft, The Netherlands

    Cynthia Liem

  4. University of Amsterdam, Amsterdam, The Netherlands

    Marcel Worring

  5. Reykjavik University, Reykjavik, Iceland

    Björn Þór Jónsson

  6. Microsoft Research Lab – Asia, Beijing, China

    Bei Liu

  7. The University of Tokyo, Tokyo, Japan

    Yoko Yamakata

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dai, S. et al. (2024). Improving Small License Plate Detection with Bidirectional Vehicle-Plate Relation. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14555. Springer, Cham. https://doi.org/10.1007/978-3-031-53308-2_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53308-2_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53307-5

  • Online ISBN: 978-3-031-53308-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation