Skip to main content
Springer Nature Link
Log in

HD number plate localization and character segmentation on the Zynq heterogeneous SoC

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Automatic number plate recognition (ANPR) systems have become widely used in safety, security, and commercial aspects. A typical ANPR system consists of three main stages: number plate localization (NPL), character segmentation (CS), and optical character recognition (OCR). In recent years, to provide a better recognition rate, high-definition (HD) cameras have started to be used. However, most known techniques for standard definition (SD) are not suitable for real-time HD image processing due to the computationally intensive cost of processing several-folds more of image pixels, particularly in the NPL stage. In this paper, algorithms suitable for hardware implementation for NPL and CS stages of an HD ANPR system are presented. Software implementation of the algorithms was carried on as a proof of concept, followed by hardware implementation on a heterogeneous system-on-chip (SoC) device that contains an ARM processor and a field-programmable gate array (FPGA). Heterogeneous implementation of these stages has shown that this HD NPL algorithm can localize a number plate in 16.17 ms, with a success rate of 98.0%. The CS algorithm can then segment the detected plate in 0.59 ms, with a success rate of 99.05%. Both stages utilize only 21% of the available on-chip configurable logic blocks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Reproduced with permission from [20]

Fig. 2

Reproduced with permission from [20]

Fig. 3

Reproduced with permission from [20]

Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Jeffrey, Z., Ramalingam, S.: High definition licence plate detection algorithm. In: 2012 Proceedings of IEEE Southeastcon, FL, pp. 1–6 (2012). https://doi.org/10.1109/secon.2012.6196912

  2. Zhai, X., Bensaali, F.: Standard definition ANPR system on FPGA and an approach to extend it to HD. In: 7th IEEE GCC Conference and Exhibition, pp. 214–219, Qatar (Nov 2013). https://doi.org/10.1109/ieeegcc.2013.6705778

  3. Du, S., Ibrahim, M., Shehata, M., Badawy, W.: Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans. Circuits Syst. Video Technol. 23(2), 311–325 (2013). https://doi.org/10.1109/TCSVT.2012.2203741

    Article  Google Scholar 

  4. Zhai, X., Dissertation, PhD: Automatic number plate recognition on FPGA. University of Hertfordshire, Hertfordshire (2013)

    Book  Google Scholar 

  5. Arth, C., Leistner, C., Bischof, H.: TRIcam: an embedded platform for remote traffic surveillance. In: Proceedings of IEEE Computer Vision and Pattern Recognition Conference, 2006, pp. 125–134 (2006). https://doi.org/10.1109/cvprw.2006.208

  6. Xilinx: Zynq-7000 All Programable SoC. Available http://www.xilinx.com/products/silicondevices/soc/zynq-7000.html (2016). Accessed on Apr 2016

  7. Arth, C., Limberger, F., Bischof, H.: Real-time license plate recognition on an embedded DSP-platform. Presented at the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007). https://doi.org/10.1109/cvpr.2007.383412

  8. Kanamori, T., Amano, H., Arai, M., Konno, D., Nanba, T., Ajioka, Y.: Implementation and evaluation of a high speed license plate recognition system on an FPGA. In: 7th International Conference on Computer and Information Technology, 2007, pp. 567–572 (2007). https://doi.org/10.1109/cit.2007.167

  9. Zhai, X., Bensaali, F., Ramalingam, S.: Improved number plate localisation algorithm and its efficient field programmable gate arrays implementation. IET Circuits Dev. Syst. 7(2), 93–103 (2013). https://doi.org/10.1049/iet-cds.2012.0064

    Article  Google Scholar 

  10. Bulan, O., Kozitsky, V., Ramesh, P., Shreve, M.: Segmentation- and annotation-free license plate recognition with deep localization and failure identification. IEEE Trans. Intell. Transp. Syst. 18(9), 2351–2363 (2017). https://doi.org/10.1109/TITS.2016.2639020

    Article  Google Scholar 

  11. Anagnostopoulos, I., Anagnostopoulos, I., Psoroulas, I., Loumos, V., Kayafas, E.: License plate recognition from still images and video sequences: a survey. IEEE Trans. Intell. Transp. Syst. 9(3), 377–391 (2008). https://doi.org/10.1109/tits.2008.922938

    Article  Google Scholar 

  12. Zhai, X., Bensaali, F.: Improved number plate character segmentation algorithm and its efficient FPGA implementation. J. Real Time Image Process. 10(1), 91–103 (2015). https://doi.org/10.1007/s11554-012-0258-5

    Article  Google Scholar 

  13. Schlesinger, M.I., Hlavác, V.: Ten Lectures on Statistical and Structural Pattern Recognition. Springer, Berlin (2002). ISBN 978-1-4020-0642-5

    Book  Google Scholar 

  14. Caner, H., Gecin, H.S., Alkar, A.Z.: Efficient embedded neural network based license plate recognition. IEEE Trans. Veh. Technol. 57, 2675–2683 (2008). https://doi.org/10.1109/TVT.2008.915524

    Article  Google Scholar 

  15. Youngwoo, Y., Kyu-Dae, B., Hosub, Y., Jaehong, K.: Blob extraction based character segmentation method for automatic license plate recognition system. In: IEEE International Conference on Systems, Man, and Cybernetics, 2011, pp. 2192–2196 (2011). https://doi.org/10.1109/icsmc.2011.6084002

  16. Liu, Y., Huang, H., Cao, J., Huang, T.: Convolutional neural networks-based intelligent recognition of Chinese license plates. J. Soft Comput. (2017). https://doi.org/10.1007/s00500-017-2503-0

    Article  Google Scholar 

  17. Kim, K.-B., Jang, S.-W., Kim, C.-K.: Recognition of car license plate by using dynamical thresholding method and enhanced neural networks. In: Petkov N., Westenberg M. (eds.) Computer Analysis of Images and Patterns, vol. 2756. Springer, Berlin (2003). https://doi.org/10.1007/978-3-540-45179-2_39

    Chapter  Google Scholar 

  18. Cui, Y., Huang, Q.: Extracting characters of license plates from video sequences. Mach. Vis. Appl. 10(5–6), 308–320 (1998). https://doi.org/10.1007/s001380050081

    Article  Google Scholar 

  19. Anagnostopoulos, C.N.E., Anagnostopoulos, I.E., Loumos, V., Kayafas, E.: A license plate recognition algorithm for intelligent transportation system applications. IEEE Trans. Intell. Transp. Syst. 7, 377–392 (2006). https://doi.org/10.1109/TITS.2006.880641

    Article  Google Scholar 

  20. Hommos, O., Al-Qahtani, A., Farhat, A., Al-Zawqari, A., Bensaali, F., Amira, A., Zhai, X.: HD Qatari ANPR system. In: CIICS16, Dubai, United Arab Emirates, pp. 1–5 (2016). https://doi.org/10.1109/iccsii.2016.7462420

  21. Gonzalez, R., Woods, R.: Digital Image Processing. Pearson, New Delhi (2014). ISBN 978-0-13-335672-4

    Google Scholar 

  22. Ismail, A., Hassaballah, M.: Image feature detectors and descriptors: foundations and applications. In: Studies in Computational Intelligence Series. Springer, Berlin (2016). https://doi.org/10.1007/978-3-319-28854-3

    Google Scholar 

  23. Zedboard.org.: Community-based site. Available http://zedboard.org/. Accessed on Dec 2016

  24. Xilinx: Accelerating OpenCV Applications with Zynq-7000 all programable SoC using Vivado HLS video libraries. Available http://www.xilinx.com/support/documentation/application_notes/xapp1167.pdf. Accessed on Apr 2016

  25. Linaro: Leading collaboration in the ARM Ecosystem, Linaro. Available http://www.linaro.org/. Accessed on Dec 2016

  26. Chang, S.L., Chen, L.S., Chung, Y.C., Chen, S.W.: Automatic license plate recognition. IEEE Trans. Intell. Transp. Syst. 5, 42–53 (2004). https://doi.org/10.1109/TITS.2004.825086

    Article  Google Scholar 

  27. Clemens, A., Florian, L., Horst, B.: Real-time license plate recognition on an embedded DSP-platform. In: Proceedings of IEEE Computer Vision and Pattern Recognition Conference, pp. 1–8 (2007). https://doi.org/10.1109/cvpr.2007.383412

Download references

Acknowledgements

This publication was made possible by UREP Grant #17-138-2-037 from the Qatar National Research Fund (a member of Qatar foundation). The statements made herein are solely the responsibility of the authors. The authors would also like to thank security services at Qatar University for providing the data used in this paper.

Author information

Author notes

  1. Ali Al-Zawqari and Omar Hommos have equally contributed to this work.

Authors and Affiliations

  1. College of Engineering, Qatar University, Doha, Qatar

    Ali Al-Zawqari, Omar Hommos, Abdulhadi Al-Qahtani, Ali A. H. Farhat, Faycal Bensaali & Abbes Amira

  2. Department of Electronics, Computing and Mathematics, University of Derby, Derby, UK

    Xiaojun Zhai

Authors
  1. Ali Al-Zawqari
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Omar Hommos
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Abdulhadi Al-Qahtani
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ali A. H. Farhat
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Faycal Bensaali
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Xiaojun Zhai
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Abbes Amira
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ali Al-Zawqari.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Zawqari, A., Hommos, O., Al-Qahtani, A. et al. HD number plate localization and character segmentation on the Zynq heterogeneous SoC. J Real-Time Image Proc 16, 2351–2365 (2019). https://doi.org/10.1007/s11554-017-0747-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-017-0747-7

Keywords