Skip to main content
SpringerLink
Log in

Number plate recognition from enhanced super-resolution using generative adversarial network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Identification and recognition of number plate is very difficult from low resolution images due to poor boundary and contrast. Our goal is to identify the digits from a low-quality number plate image correctly, but correct detection was exceedingly difficult in some cases due to the low-resolution image. Another goal of this paper was to upscale the image from a very low resolution to high resolution to recover helpful information to improve the accuracy of number plate detection and recognition. We have used Enhanced- Super-Resolution with Generative Adversarial Network (SRGAN). We modified native Dense Blocks of the Generative Adversarial Network with a Residual in Residual Dense Block model. In addition to Convolutional Neural Networks for thresholding. We also used a Rectified Linear Unit (ReLU) activation layer. The plate image is then used for segmentation using the OCR model for detection and recognizing the characters in the number plates. The Optical character recognition (OCR) model reaches an average accuracy of 84% for high resolution, whereas the accuracy is 4% - 7% for low resolution. The model’s accuracy increases with the resolution enhancement of the plate images. ESRGAN provides better enhancement of low-resolution images than SRGAN and Pro-SRGAN, which the OCR model validates. The accuracy significantly increased digit/alphabet detection in the number plate than the original low-resolution image when converted to a high-resolution image using ESRGAN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Amit Y, Geman D, Fan X (2004) A coarse-to-fine strategy formulticlass shape detection. IEEE Trans Pattern Anal Mach Intell 26:1606–1621

    Article  Google Scholar 

  2. Car License Plates Dataset (https://makeml.app/datasets/cars-license-plates visited on 3rd July, 2022)

  3. Chatterjee S, Dutta RK, Ganguly D, Chatterjee K, Roy S (2020) Bengali handwritten character classification using transfer learning on deep convolutional network. In: Tiwary U, Chaudhury S (eds) Intelligent human computer interaction. IHCI 2019. Lecture notes in computer science(), vol 11886. Springer, Cham https://doi.org/10.1007/978-3-030-44689-5_13

  4. Chen H, He X, Qing L, Wu Y, Ren C, Sheriff RE, Zhu C (2022) Real-world single image super-resolution: a brief review. Inf Fusion 79:124–145

    Article  Google Scholar 

  5. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: ECCV

  6. El-Shal IH, Fahmy OM, Elattar MA (2022) License plate image analysis empowered by generative adversarial neural networks (GANs). IEEE Access 10:30846–30857. https://doi.org/10.1109/ACCESS.2022.3157714

    Article  Google Scholar 

  7. Girdher H, Sharma H, Gupta A (2022) Comprehensive survey on devanagari OCR. Available at SSRN 4033489

  8. Karthick K et al (2019) Consumer service number recognition using template matching algorithm for improvements in ocr based energy consumption billing. ICIC Exp Lett Part B: Appl ICIC Int 10.10

  9. Keipour A et al (2022) Omnifont persian OCR system using primitives. arXiv preprint arXiv:2202.06371

  10. Khan RA, Meyer A, Konik H, Bouakaz S (2019) Saliency-based framework for facial expression recognition. Front Comput Sci 13(1):183–198

    Article  Google Scholar 

  11. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: CVPR

  12. Kumar M, Jindal SR, Jindal MK, Lehal GS (2018) Improved recognition results of medieval handwritten Gurmukhi manuscripts using boosting and bagging methodologies. Neural Process Lett 50:43–56

    Article  Google Scholar 

  13. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: CVPR

  14. Laroca R et al (2022) On the cross-dataset generalization for license plate recognition. arXiv preprint arXiv:2201.00267

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  16. Lin M, Liu L, Wang F, Li J, Pan J (2021) License plate image reconstruction based on generative adversarial networks. Remote Sens 13(15):3018. https://doi.org/10.3390/rs13153018

    Article  Google Scholar 

  17. Onim MD et al (2022) BLPnet: A new DNN model and Bengali OCR engine for automatic license plate recognition. arXiv preprint arXiv:2202.12250

  18. Qiao C, Li D, Guo Y, Liu C, Jiang T, Dai Q, Li D (2021) Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat Methods 18(2):194–202

    Article  Google Scholar 

  19. Radwan MA, Khalil MI, Abbas HM (2018) Neural networks pipeline for offline machine printed Arabic OCR. Neural Process Lett 48(2):769–787

    Article  Google Scholar 

  20. Rahati S, Morvejian R, Kazemi EM, Kazem FM (2008) Vehicle recognition using contourlet transform and SVM. Proceedings of the Fifth International Conference on Information Technology

  21. Rostami M, Berahmand K, Nasiri E, Forouzandeh S (2021) Review of swarm intelligence-based feature selection methods. Eng Appl Artif Intell 100:104210

    Article  Google Scholar 

  22. Rostami M, Forouzandeh S, Berahmand K, Soltani M (2020) Integration of multi-objective PSO based feature selection and node centrality for medical datasets. Genomics 112(6):4370–4384

    Article  Google Scholar 

  23. Rostami M, Forouzandeh S, Berahmand K, Soltani M, Shahsavari M, Oussala M (2022) Gene selection for microarray data classification via multi-objective graph theoretic-based method. Artif Intell Med 123:102228

    Article  Google Scholar 

  24. Roy S, Bhattacharyya D, Bandyopadhyay SK, Kim TH (2017) An improved brain MR image binarization method as a preprocessing for abnormality detection and features extraction. Front Comput Sci 11:717–727. https://doi.org/10.1007/s11704-016-5129-y

    Article  Google Scholar 

  25. Roy S, Whitehead TD, Li S, Ademuyiwa FO, Wahl RL, Dehdashti F, Shoghi KI (2022) Co-clinical FDG-PET radiomic signature in predicting response to neoadjuvant chemotherapy in triple-negative breast cancer. Eur J Nucl Med Mol Imaging 49(2):550–562

    Article  Google Scholar 

  26. Roy S, Whitehead TD, Quirk JD, Salter A, Ademuyiwa FO, Li S, An H, Shoghi KI (2020) Optimal co-clinical radiomics: sensitivity of radiomics features to tumor volume, image noise and resolution in co-clinical T1-weighted and T2-weighted magnetic resonance imaging. EBioMedicine, The Lancet, volume 59, 102963

  27. Sabóia CMG (2022) Brazilian mercosur license plate detection and recognition using haar cascade and tesseract OCR on synthetic imagery. International Conference on Intelligent Systems Design and Applications. Springer, Cham

  28. SaimaRafique, Iqbal M, Habib HA (2009) Space invariant vehicle recognition for toll plaza monitoring and auditing system. Multitopic Conference. INMIC 2009, IEEE 13th International, pp 1–6

  29. Salma MS, Rauf ur R, Khan MG, Zulfiqar A, Bhatti MT (2021) Development of ANPR framework for Pakistani vehicle number plates using object detection and OCR. Complexity 2021, Article ID 5597337, 14 pages. https://doi.org/10.1155/2021/5597337

  30. Silva SM, Jung CR (2020) Real-time license plate detection and recognition using deep convolutional neural networks. J Vis Commun Image Represent 102773. https://doi.org/10.1016/j.jvcir.2020.102773

  31. Singh S, Sharma A, Chauhan VK (2021) Online handwritten Gurmukhi word recognition using fine-tuned Deep Convolutional Neural Network on offline features. Mach Learn Applic 5:100037

    Google Scholar 

  32. Srilekha B, Kiran KVD, Padyala VVP (2022) Detection of license plate numbers and identification of non-helmet riders using yolo v2 and OCR method. 2022 International Conference on Electronics and Renewable Systems (ICEARS). IEEE

  33. Su H, Tang L, Wu Y, Tretter D, Zhou J (2011) Spatially adaptive block-based super-resolution. IEEE Trans Image Process 21(3):1031–1045

    MathSciNet  MATH  Google Scholar 

  34. Vu X-S et al (2021) Mc-ocr challenge: Mobile-captured image document recognition for vietnamese receipts. 2021 RIVF International Conference on Computing and Communication Technologies (RIVF). IEEE

  35. Wang R, Fujii Y, Popat AC (2022) Post-ocr paragraph recognition by graph convolutional networks. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

  36. Wang X, Xie L, Dong C, Shan Y (2021) Real-esrgan: training real-world blind super-resolution with pure synthetic data. In proceedings of the IEEE/CVF international conference on computer vision (pp 1905–1914)

  37. Zhai X, Bensaali F, Sotudeh R (2012) OCR-based neural network for ANPR. In IEEE, Pp1

  38. Zhang K, Sun M, Han X, Yuan X, Guo L, Liu T (2017) Residual networks of residual networks: multilevel residual networks. IEEE Trans Circ Syst Video Technol

  39. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: CVPR

Download references

Source code and data

The complete source code and data link is available at: https://github.com/bad-eastwind/Super-Resolution-GAN-

Author information

Authors and Affiliations

  1. Artificial Intelligence & Data Science, Jio Institute, Navi Mumbai, Maharastra, 410206, India

    Anwesh Kabiraj, Debojyoti Pal & Sudipta Roy

  2. Information Technology, Government College of Engineering and Leather Technology, Sector-III, Bidhannagar, Kolkata, West Bengal, 7000983, India

    Anwesh Kabiraj

  3. Computer Science and Engineering, Government College of Engineering and Leather Technology, Sector-III, Bidhannagar, Kolkata, West Bengal, 7000983, India

    Debojyoti Pal & Debayan Ganguly

  4. Computer Science and Engineering, Government College of Engineering and Ceramic Technology, Beleghata, Kolkata, West Bengal, 700010, India

    Kingshuk Chatterjee

Authors
  1. Anwesh Kabiraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debojyoti Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debayan Ganguly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kingshuk Chatterjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sudipta Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudipta Roy.

Ethics declarations

Conflict of interest

Authors have no conflict of interest to declare.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kabiraj, A., Pal, D., Ganguly, D. et al. Number plate recognition from enhanced super-resolution using generative adversarial network. Multimed Tools Appl 82, 13837–13853 (2023). https://doi.org/10.1007/s11042-022-14018-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14018-0

Keywords

Search

Navigation