Skip to main content
SpringerLink
Log in

Synthetic image generation for training deep learning-based automated license plate recognition systems on the Brazilian Mercosur standard

  • Published:
Design Automation for Embedded Systems Aims and scope Submit manuscript

Abstract

License plates are the primary source of vehicle identification data used in a wide range of applications including law enforcement, electronic tolling, and access control amongst others. License plate detection (LPD) is a critical process in automatic license plate recognition (ALPR) that reduces complexity by delimiting the search space for subsequent ALPR stages. It is complicated by unfavourable factors including environmental conditions, occlusion, and license plate variation. As such, it requires training models on substantial volumes of relevant images per use case. In 2018, the new Mercosur standard came in to effect in four South American countries. Access to large volumes of actual Mercosur license plates with sufficient presentation variety is a significant challenge for training supervised models for LPD, thereby adversely impacting the efficacy of ALPR in Mercosur countries. This paper presents a novel license plate embedding methodology for generating large volumes of accurate Mercosur license plate images sufficient for training supervised LPD. We validate this methodology with a deep learning-based ALPR using a convolutional neural network trained exclusively with synthetic data and tested with real parking lot and traffic camera images. Experiment results achieve detection accuracy of 95% and an average running time of 40 ms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Mercosur is sometimes referred to as Mercosul in Portuguese (Brazil) or emby emuha in Guarani (Paraguay).

  2. Venezuela is currently suspended.

References

  1. Amorim VH, Carvalho BM, Thomé AC (2019) A modified Viola–Jones detector for low-cost localization of car plates. In: International symposium on visual computing. Springer, pp 529–540

  2. Anagnostopoulos CNE (2014) License plate recognition: a brief tutorial. IEEE Intell Transp Syst Mag 6(1):59–67

    Article  Google Scholar 

  3. Anagnostopoulos CNE, Anagnostopoulos IE, Psoroulas ID, Loumos V, Kayafas E (2008) License plate recognition from still images and video sequences: a survey. IEEE Trans Intell Transp Syst 9(3):377–391

    Article  Google Scholar 

  4. Asif MR, Chun Q, Hussain S, Fareed MS, Khan S (2017) Multinational vehicle license plate detection in complex backgrounds. J Vis Commun Image Represent 46:176–186

    Article  Google Scholar 

  5. Babu CNK, Nallaperumal K (2008) An efficient geometric feature based license plate localization and recognition. Int J Imaging Sci Eng 2(2):189–194

    Google Scholar 

  6. Baerwald JE(1960) The functions and design of motor vehicle license plates. Tech. rep., University of Illinois at Urbana Champaign, College of Engineering

  7. Barbosa IB, Cristani M, Caputo B, Rognhaugen A, Theoharis T (2018) Looking beyond appearances: synthetic training data for deep cnns in re-identification. Comput Vis Image Underst 167:50–62

    Article  Google Scholar 

  8. Barron E, Slomowitz A (2013)Massachusetts should facilitate not inhibit law enforcement use of license plate data. Tech. rep., Digital Recognition Network and Vigilant Solutions

  9. Bjrklund T, Fiandrotti A, Annarumma M, Francini G, Magli E (2019) Robust license plate recognition using neural networks trained on synthetic images. Pattern Recognit 93:134–146. https://doi.org/10.1016/j.patcog.2019.04.007

    Article  Google Scholar 

  10. Bradski G (2000) The OpenCV Library. Dr. Dobb’s Journal of Software Tools

  11. Bulan O, Kozitsky V, Ramesh P, Shreve M (2017) Segmentation- and annotation-free license plate recognition with deep localization and failure identification. IEEE Trans Intell Transp Syst 18(9):2351–2363. https://doi.org/10.1109/TITS.2016.2639020

    Article  Google Scholar 

  12. Chang SL, Chen LS, Chung YC, Chen SW (2004) Automatic license plate recognition. IEEE Trans Intell Transp Syst 5(1):42–53

    Article  Google Scholar 

  13. Chung CK, Hsieh YK, Wang YH et al (2016) Aware and smart member card: RFID and license plate recognition systems integrated applications at parking guidance in shopping mall. In: 2016 Eighth international conference on advanced computational intelligence (ICACI). IEEE, pp 253–256

  14. Conselho Nacional de Trnsito (CONTRAN): Resolu n 729 (2018). http://www.denatran.gov.br/images/Resolucoes/Resolucao7292018consolidada.pdf. Accessed 30 June 2019

  15. Costa DG, Silva I, Guedes LA, Portugal P, Vasques F (2014) Selecting redundant nodes when addressing availability in wireless visual sensor networks. In: 2014 12th IEEE international conference on industrial informatics (INDIN), pp 130–135. https://doi.org/10.1109/INDIN.2014.6945496

  16. Costa DG, Silva I, Guedes LA, Vasques F, Portugal P (2015) Optimal sensing redundancy for multiple perspectives of targets in wireless visual sensor networks. In: 2015 IEEE 13th international conference on industrial informatics (INDIN), pp 185–190 . https://doi.org/10.1109/INDIN.2015.7281732

  17. Dlagnekov L (2004) License plate detection using adaboost. Computer Science and Engineering Department, San Diego

    Google Scholar 

  18. Du S, Ibrahim M, Shehata M, Badawy W (2012) Automatic license plate recognition (ALPR): a state-of-the-art review. IEEE Trans Circuits Syst Video Technol 23(2):311–325

    Article  Google Scholar 

  19. Du S, Ibrahim M, Shehata M, Badawy W (2013) Automatic license plate recognition (alpr): a state-of-the-art review. IEEE Trans Circuits Syst Video Technol 23(2):311–325

    Article  Google Scholar 

  20. Gierlack K, Williams S, LaTourrette T, Anderson JM, Mayer LA (2014) License plate readers for law enforcement: opportunities and obstacles. Rand Corporation, Santa Monica

    Google Scholar 

  21. Gou C, Wang K, Yao Y, Li Z (2016) Vehicle license plate recognition based on extremal regions and restricted boltzmann machines. IEEE Trans Intell Transp Syst 17(4):1096–1107

    Article  Google Scholar 

  22. Greati VR, Ribeiro VCT, da Silva IMD, de Medeiros Martins A (2017)A Brazilian license plate recognition method for applications in smart cities. In: 2017 IEEE first summer school on smart cities (S3C), pp 43–48. https://doi.org/10.1109/S3C.2017.8501395

  23. Gupta A, Vedaldi A, Zisserman A (2016)Synthetic data for text localisation in natural images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2315–2324

  24. Hinterstoisser S, Lepetit V, Wohlhart P, Konolige K (2019) On pre-trained image features and synthetic images for deep learning. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 11129 LNCS, pp 682–697. https://doi.org/10.1007/978-3-030-11009-3_42

  25. Hongliang B, Changping L (2004)A hybrid license plate extraction method based on edge statistics and morphology. In: Proceedings of the 17th international conference on pattern recognition. ICPR 2004, vol 2. IEEE, pp 831–834

  26. Hongping H, Yanping B(2011) A kind of license plate location based on mathematical morphology and edge detection. In: Proceedings of 2011 international conference on electronic and mechanical engineering and information technology, vol 5. IEEE, pp 2291–2294

  27. Hsieh JW, Yu SH, Chen YS (2002) Morphology-based license plate detection from complex scenes. In: Object recognition supported by user interaction for service robots, vol 3. IEEE, pp 176–179

  28. Ijaz S, Shah MA, Khan A, Ahmed M (2016) Smart cities: a survey on security concerns. Int J Adv Comput Sci Appl 7(2):612–625

    Google Scholar 

  29. Ismagilova E, Hughes L, Dwivedi YK, Raman KR (2019) Smart cities: advances in research an information systems perspective. Int J Inf Manag 47:88–100. https://doi.org/10.1016/j.ijinfomgt.2019.01.004

    Article  Google Scholar 

  30. Izidio DM, Ferreira AP, Medeiros HR, Barros ENDS (2019) An embedded automatic license plate recognition system using deep learning. Design Automation for Embedded Systems, pp 1–21

  31. Izidio DMF, Ferreira APA, Barros ENS (2018) An embedded automatic license plate recognition system using deep learning. In: 2018 VIII Brazilian symposium on computing systems engineering (SBESC), pp 38–45. https://doi.org/10.1109/SBESC.2018.00015

  32. Jia W, Zhang H, He X (2007) Region-based license plate detection. J Netw Comput Appl 30(4):1324–1333

    Article  Google Scholar 

  33. Johnson-Roberson M, Barto C, Mehta R, Sridhar SN, Rosaen K, Vasudevan R (2016)Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks? arXiv preprint arXiv:1610.01983

  34. Kamat V, Ganesan S (1995) An efficient implementation of the hough transform for detecting vehicle license plates using dsp’s. In: Proceedings real-time technology and applications symposium. IEEE, pp 58–59

  35. Kim KI, Jung K, Kim JH (2002) Color texture-based object detection: an application to license plate localization. In: International workshop on support vector machines. Springer, pp 293–309

  36. Laroca R, Severo E, Zanlorensi LA, Oliveira LS, Gonçalves GR, Schwartz WR, Menotti D (2018) A robust real-time automatic license plate recognition based on the Yolo detector. In: 2018 international joint conference on neural networks (IJCNN). IEEE, pp 1–10

  37. Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and lstms. arXiv preprint arXiv:1601.05610

  38. Li H, Wang P, You M, Shen C (2018) Reading car license plates using deep neural networks. Image Vis Comput 72:14–23

    Article  Google Scholar 

  39. Matas J, Galambos C, Kittler J (2000) Robust detection of lines using the progressive probabilistic hough transform. Comput Vis Image Underst 78(1):119–137. https://doi.org/10.1006/cviu.1999.0831

    Article  Google Scholar 

  40. Official Web Portal of the Government of Brasil: Brasil ter modelo de placas de veculos unificado com o mercosul. http://www.brasil.gov.br/governo/2014/10/brasil-tera-modelo-de-placas-de-veiculos-unificado-com-o-mercosul. Accessed 30 June 2019

  41. Oliveira MA (2014) Mercosul cria placa nica para veculos . http://g1.globo.com/mundo/noticia/2010/12/mercosul-cria-placa-unica-para-veiculos.html. Accessed 30 June 2019

  42. OpenCV: Opencv modules (2019) https://docs.opencv.org/4.1.0/. Accessed 30 June 2019

  43. Pellicer S, Santa G, Bleda AL, Maestre R, Jara AJ, Skarmeta AG (2013) A global perspective of smart cities: a survey. IEEE. https://doi.org/10.1109/IMIS.2013.79. http://ieeexplore.ieee.org/document/6603712/

  44. Peng Y, Xu M, Jin JS, Luo S, Zhao G (2011) Cascade-based license plate localization with line segment features and Haar-like features. In: 2011 sixth international conference on image and graphics. IEEE, pp 1023–1028

  45. Prabhakar P, Anupama P (2014) A novel design for vehicle license plate detection and recognition. In: Second international conference on current trends in engineering and technology-ICCTET 2014. IEEE, pp 7–12

  46. Redmon J (2019) Darknet: open source neural networks in c. http://pjreddie.com/darknet/. Accessed 30 June (2013–2016)

  47. Redmon J, Farhadi A (2018) Yolov3: an incremental improvement. CoRR abs/1804.02767. http://arxiv.org/abs/1804.02767

  48. Ribeiro V, Greati V, Bezerra A, Silvano G, Silva I, Endo PT, Lynn T (2019) Brazilian mercosur license plate detection: a deep learning approach relying on synthetic imagery. In: 2019 IX Brazilian symposium on computing systems engineering (SBESC). IEEE, pp 1–8

  49. Roberts DJ, Casanova M (2012) Automated license plate recognition (ALPR) use by law enforcement: Policy and operational guide. Tech. rep, National Criminal Justice Reference Service

  50. Rozantsev A, Lepetit V, Fua P (2015) On rendering synthetic images for training an object detector. Comput Vis Image Underst 137:24–37

    Article  Google Scholar 

  51. Sarfraz M, Ahmed MJ, Ghazi SA(2003) Saudi Arabian license plate recognition system. In: 2003 international conference on geometric modeling and graphics, 2003. Proceedings. IEEE, pp 36–41

  52. Seibel H, Goldenstein S, Rocha A (2017) Eyes on the target: super-resolution and license-plate recognition in low-quality surveillance videos. IEEE Access 5:20020–20035. https://doi.org/10.1109/ACCESS.2017.2737418

    Article  Google Scholar 

  53. Selmi Z, Ben Halima M, Alimi AM (2017) Deep learning system for automatic license plate detection and recognition. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR), vol 01, pp 1132–1138. https://doi.org/10.1109/ICDAR.2017.187

  54. Serrano JAR, Bala R, Perronnin F, Saunders CJ, Zhao Y (2013) Methods and systems for improved license plate signature matching by similarity learning on synthetic images

  55. Silva IMD, Guedes LA, Vasques F (2008) Performance evaluation of a compression algorithm for wireless sensor networks in monitoring applications. In: 2008 IEEE international conference on emerging technologies and factory automation, pp 672–678. https://doi.org/10.1109/ETFA.2008.4638468

  56. Silva SM, Jung CR (2017) Real-time Brazilian license plate detection and recognition using deep convolutional neural networks. In: 2017 30th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI). IEEE, pp 55–62

  57. Sonavane K, Soni B, Majhi U (2015) Survey on automatic number plate recognition (anr). Int J Comput Appl 125(6):1–4

    Google Scholar 

  58. Tan JL, Abu-Bakar SA, Mokji MM (2013) License plate localization based on edge-geometrical features using morphological approach. In: 2013 IEEE international conference on image processing. IEEE, pp 4549–4553

  59. Tremblay J, Prakash A, Acuna D, Brophy M, Jampani V, Anil C, To T, Cameracci E, Boochoon S, Birchfield S (2018) Training deep networks with synthetic data: Bridging the reality gap by domain randomization. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 969–977

  60. Vicente AN, Pedrini H(2016) A learning-based single-image super-resolution method for very low quality license plate images. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), October, pp. 000515–000520. IEEE. https://doi.org/10.1109/SMC.2016.7844291. http://ieeexplore.ieee.org/document/7844291/

  61. Wang J, Huang H, Qian X, Cao J, Dai Y (2018) Sequence recognition of Chinese license plates. Neurocomputing 317:149–158. https://doi.org/10.1016/j.neucom.2018.08.023

    Article  Google Scholar 

  62. Xiao J, Hays J, Ehinger KA, Oliva A, Torralba A(2010) Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 3485–3492. https://doi.org/10.1109/CVPR.2010.5539970

  63. Yin C, Xiong Z, Chen H, Wang J, Cooper D, David B (2015) A literature survey on smart cities. Sci China Inf Sci 58(10):1–18. https://doi.org/10.1007/s11432-015-5397-4

    Article  Google Scholar 

  64. You F, Zhao Y, Wang X (2019) Combination of CNN with GRU for plate recognition. J Phys Conf Ser 1187:032008

    Article  Google Scholar 

  65. Yuan Y, Zou W, Zhao Y, Wang X, Hu X, Komodakis N (2017) A robust and efficient approach to license plate detection. IEEE Trans Image Process 26(3):1102–1114

    Article  MathSciNet  Google Scholar 

  66. Zhao Y, Gu J, Liu C, Han S, Gao Y, Hu Q(2010) License plate location based on Haar-like cascade classifiers and edges. In: 2010 second WRI global congress on intelligent systems, vol 3. IEEE, pp 102–105

  67. Zheng D, Zhao Y, Wang J (2005) An efficient method of license plate location. Pattern Recognit Lett 26(15):2431–2438

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics and Applied Mathematics, Federal University of Rio Grande do Norte, Natal, Rio Grande do Norte, Brazil

    Gilles Silvano, Vinícius Ribeiro, Vitor Greati, Aguinaldo Bezerra & Ivanovitch Silva

  2. University of Pernambuco, Pernambuco, Brazil

    Patricia Takako Endo

  3. Irish Institute of Digital Business, Dublin City University, Dublin, Ireland

    Theo Lynn

Authors
  1. Gilles Silvano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vinícius Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vitor Greati
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aguinaldo Bezerra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ivanovitch Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Patricia Takako Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Theo Lynn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gilles Silvano.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silvano, G., Ribeiro, V., Greati, V. et al. Synthetic image generation for training deep learning-based automated license plate recognition systems on the Brazilian Mercosur standard. Des Autom Embed Syst 25, 113–133 (2021). https://doi.org/10.1007/s10617-020-09241-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10617-020-09241-7

Keywords

Search

Navigation