Abstract
The goal of license plate recognition (LPR) is to read the license plate characters. Due to image degradation, there are many difficulties in the way of achieving this goal. In this paper, the proposed method recognizes the license plate characters without employing the traditional segmentation and binarization techniques. This method uses a deep learning algorithm and tries to achieve better learning experience by engaging a multi-task learning algorithm based on sharing features. The features of license plate characters are extracted by a deep encoder-decoder network, and transferred to 8 parallel classifiers for recognition. To evaluate the current work, a database of 11,000 license plate images, collected from a currently working surveillance system installed on a dual carriageway, is employed. The proposed method achieved the correct character recognition rate of 96% for 4000 test images that is acceptable in comparison to the competing methods.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig7_HTML.jpg)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig8_HTML.jpg)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig11_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig12_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig13_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig14_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-019-08416-0/MediaObjects/11042_2019_8416_Fig15_HTML.png)
Similar content being viewed by others
Notes
t-distributed stochastic neighbor embedding (t-SNE)
References
Abolghasemi V, Ahmadyfard A (2009) An edge-based color-aided method for license plate detection. Image Vis Comput 27:1134–1142. https://doi.org/10.1016/j.imavis.2008.10.012
Al-Rfou R, Alain G, Almahairi A, Angermueller C, Bahdanau D, Ballas N, Bastien F, Bayer J, Belikov A, Belopolsky A, Bengio Y, Bergeron A, Bergstra J, Bisson V, Bleecher Snyder J, Bouchard N, Boulanger-Lewandowski N, Bouthillier X, de Brébisson A, Breuleux O, Carrier P-L, Cho K, Chorowski J, Christiano P, Cooijmans T, Côté M-A, Côté M, Courville A, Dauphin YN, Delalleau O, Demouth J, Desjardins G, Dieleman S, Dinh L, Ducoffe M, Dumoulin V, Ebrahimi Kahou S, Erhan D, Fan Z, Firat O, Germain M, Glorot X, Goodfellow I, Graham M, Gulcehre C, Hamel P, Harlouchet I, Heng J-P, Hidasi B, Honari S, Jain A, Jean S, Jia K, Korobov M, Kulkarni V, Lamb A, Lamblin P, Larsen E, Laurent C, Lee S, Lefrancois S, Lemieux S, Léonard N, Lin Z, Livezey JA, Lorenz C, Lowin J, Ma Q, Manzagol P-A, Mastropietro O, McGibbon RT, Memisevic R, van Merriënboer B, Michalski V, Mirza M, Orlandi A, Pal C, Pascanu R, Pezeshki M, Raffel C, Renshaw D, Rocklin M, Romero A, Roth M, Sadowski P, Salvatier J, Savard F, Schlüter J, Schulman J, Schwartz G, Serban IV, Serdyuk D, Shabanian S, Simon É, Spieckermann S, Subramanyam SR, Sygnowski J, Tanguay J, van Tulder G, Turian J, Urban S, Vincent P, Visin F, de Vries H, Warde-Farley D, Webb DJ, Willson M, Xu K, Xue L, Yao L, Zhang S, Zhang Y (2016) Theano: A Python framework for fast computation of mathematical expressions, arXiv E-Prints. abs/1605.0. http://arxiv.org/abs/1605.02688. Accessed 2 Oct 2017
Ashtari AH, Nordin MJ, Fathy M (2014) An Iranian license plate recognition system based on color features. IEEE Trans Intell Transp Syst 15:1690–1705. https://doi.org/10.1109/TITS.2014.2304515
Bengio Y (2012) Deep learning of representations for unsupervised and transfer learning. JMLR Work Conf Proc 7:1–20
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35:1798–1828. https://doi.org/10.1109/TPAMI.2013.50
Björklund T, Fiandrotti A, Annarumma M, Francini G, Magli E (2019) Robust license plate recognition using neural networks trained on synthetic images. Pattern Recogn 93:134–146. https://doi.org/10.1016/J.PATCOG.2019.04.007
Chang SL, Chen LS, Chung YC, Chen SW (2004) Automatic license plate recognition. IEEE Trans Intell Transp Syst 5:42–53. https://doi.org/10.1109/TITS.2004.825086
Chen KN, Chen CH, Chang CC (2012) Efficient illumination compensation techniques for text images. Digit Signal Process A Rev J 22:726–733. https://doi.org/10.1016/j.dsp.2012.04.010
Chollet F (2015) Others, Keras. https://github.com/fchollet/keras
Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: Comput. Vis. Pattern Recognit. (CVPR), 2012 IEEE Conf., IEEE. https://doi.org/10.1109/CVPR.2012.6248110, p 3642–3649
CS231n convolutional neural networks for visual recognition - visualizing what ConvNets learn (n.d.). http://cs231n.github.io/understanding-cnn/
Erhan D, Courville A, Vincent P (2010) Why does unsupervised pre-training help deep learning. J Mach Learn Res 11:625–660. https://doi.org/10.1145/1756006.1756025
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press http://www.deeplearningbook.org. Accessed 10 Aug 2017
Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31. https://doi.org/10.1016/j.media.2016.05.004
He K, Zhang X, Ren S, Sun J (2016) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proc. IEEE Int. Conf. Comput. Vis. https://doi.org/10.1109/ICCV.2015.123, p 1026–1034
Hinton GE, Srivastava N, Krizhevsky A, Sutskever N, Salakhutdinov RR (2012) Improving neural net- works by preventing co-adaptation of feature detectors, arXiv:1207.0580 [cs.NE]. https://arxiv.org/abs/1207.0580, 1–5
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hsu G-S, Chen J-C, Chung Y-Z (2013) Application-oriented license plate recognition. IEEE Trans Veh Technol 62:552–561. https://doi.org/10.1109/TVT.2012.2226218
Jaderberg M, Simonyan K, Vedaldi A, Zisserman A (2016) Reading text in the wild with convolutional neural networks. Int J Comput Vis 116:1–20. https://doi.org/10.1007/s11263-015-0823-z
Jiao J, Ye Q, Huang Q (2009) A configurable method for multi-style license plate recognition. Pattern Recogn 42:358–369. https://doi.org/10.1016/j.patcog.2008.08.016
Kashef S, Nezamabadi-pour H, Rashedi E (2017) Adaptive enhancement and binarization techniques for degraded plate images. Multimed Tools Appl 77:1–19. https://doi.org/10.1007/s11042-017-5229-1
Ko M-A, Kim Y-M (2003) License plate surveillance system using weighted template matching, 32nd Appl. Imag. Pattern Recognit. Work. 2003. Proceedings. https://doi.org/10.1109/AIPR.2003.1284283, p 269–274
Ko MA, Kim YM (2004) A simple OCR method from strong perspective view(b). In: Proc. - Appl. Imag. Pattern Recognit. Work., IEEE. https://doi.org/10.1109/AIPR.2004.8, p. 235–240
Kobchaisawat T, Chalidabhongse TH (2016) A method for multi-oriented Thai text localization in natural scene images using Convolutional Neural Network. In: IEEE 2015 Int. Conf. Signal Image Process. Appl. ICSIPA 2015 - Proc. https://doi.org/10.1109/ICSIPA.2015.7412193, p. 220–225
Kocer HE, Cevik KK (2011) Artificial neural networks based vehicle license plate recognition. Procedia Comput Sci 3:1033–1037. https://doi.org/10.1016/j.procs.2010.12.169
Krizhevsky A, Hinton GE, Sutskever I (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 1–9. https://doi.org/10.1016/j.protcy.2014.09.007
Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and lSTMs, arXiv:1601.05610 [cs.CV]. 1–17. http://arxiv.org/abs/1601.05610
Li H, Shen C (2016) Reading car license plates using deep convolutional neural networks and LSTMs. http://arxiv.org/abs/1601.05610. Accessed 27 Aug 2019
Lin S-P, Wu B-F, Chiu C-C (2007) Extracting characters from real vehicle licence plates out-of-doors. IET Comput Vis 1:2–10. https://doi.org/10.1049/iet-cvi:20050132
Liu X, Kawanishi T, Wu X, Kashino K (2016) Scene text recognition with high performance CNN classifier and efficient word inference. Icassp 2016:1322–1326
Llorens D, Marzal A, Palazón V, Vilar J (2005) Car license plates extraction and recognition based on connected components analysis and HMM decoding. Springer-Verlag, Berlin Heidelb, pp 571–578. https://doi.org/10.1007/11492429_69
Naito T, Tsukada T, Yamada K, Kozuka K, Yamamoto S (2000) Robust license-plate recognition method for passing vehicles under outside environment. IEEE Trans Veh Technol 49:2309–2319. https://doi.org/10.1109/25.901900
Nejati M, Majidi A, Jalalat M (2015) License plate recognition based on edge histogram analysis and classifier ensemble. Conf Signal Process Intell Syst 48–52. https://doi.org/10.1109/SPIS.2015.7422310
Nukano T, Fukumi M, Khalid M (2004) Vehicle license plate character recognition by neural networks. In: Proc. 2004 Int. Symp. Intell. Signal Process. Commun. Syst. 2004. ISPACS 2004, IEEE. https://doi.org/10.1109/ISPACS.2004.1439164, p. 771–775
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830 http://scikit-learn.org/stable/documentation.html
Qian Y, Bi M, Tan T, Yu K (2016) Very deep convolutional neural networks for noise robust speech recognition. IEEE/ACM Trans. Audio, Speech, Lang. Process 24: 2263–2276. https://doi.org/10.1109/TASLP.2016.2602884
Rashedi E, Nezamabadi-pour H (2017) A hierarchical algorithm for vehicle license plate localization. Multimed Tools Appl 77:1–20. https://doi.org/10.1007/s11042-017-4429-z
Salahshoor M, Broumandnia A, Rastgarpour M (2014) Application of intelligent systems for Iranian license plate recognition, Iran. Conf. Intell. Syst. High. Educ. Complex, Bam, Iran. 1–6. https://doi.org/10.1109/IranianCIS.2014.6802578
Sedighi A, Vafadust M (2011) A new and robust method for character segmentation and recognition in license plate images. Expert Syst Appl 38:13497–13504. https://doi.org/10.1016/j.eswa.2011.02.030
Tunisia data set on Kaggle (visited 8/15/2019) (2019). https://www.kaggle.com/achrafkhazri/anpr-dataset-tunisian-plates-and-digits. Accessed 10 Aug 2017
van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605 http://www.jmlr.org/papers/v9/vandermaaten08a.html. Accessed 10 Aug 2017
Vázquez N, Nakano M, Pérez-Meana H (2002) Automatic system for localization and recognition of vehicle plate numbers. J Appl Res Technol 1:63–77 http://www.scielo.org.mx/pdf/jart/v1n1/v1n1a6.pdf. Accessed June 1, 2017
Vincent P, Larochelle H (2010) Stacked Denoising autoencoders: learning useful representations in a deep network with a local Denoising criterion. J Mach Learn Res 11:3371–3408. https://doi.org/10.1111/1467-8535.00290
Wang R, Sang N, Huang R, Wang Y (2014) License plate detection using gradient information and cascade detectors. Optik (Stuttg) 125:186–190. https://doi.org/10.1016/j.ijleo.2013.06.008
Xu J-F, Li S-F, Yu M-S (n.d.) Car license plate extraction using color and edge information, in: Proc. 2004 Int. Conf. Mach. Learn. Cybern. (IEEE Cat. No.04EX826), IEEE. https://doi.org/10.1109/ICMLC.2004.1380528, p. 3904–3907
Yang Y, Gao X, Yang G (2011) Study the method of vehicle license locating based on color segmentation. In: Procedia Eng., Elsevier B.V. https://doi.org/10.1016/j.proeng.2011.08.245, p 1324–1329
Yu S, Li B, Zhang Q, Liu C, Meng MQH (2015) A novel license plate location method based on wavelet transform and EMD analysis. Pattern Recogn 48:114–125. https://doi.org/10.1016/j.patcog.2014.07.027
Zheng L, He X, Samali B, Yang LT (2013) An algorithm for accuracy enhancement of license plate recognition. J Comput Syst Sci 79:245–255. https://doi.org/10.1016/j.jcss.2012.05.006
Acknowledgments
We express our especial gratitude to Mr. Milad Noorani and Taraddod Rahnama Company, Kerman, Iran for their supports and helps in providing the database.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Rakhshani, S., Rashedi, E. & Nezamabadi-pour, H. Representation learning in a deep network for license plate recognition. Multimed Tools Appl 79, 13267–13289 (2020). https://doi.org/10.1007/s11042-019-08416-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08416-0