Mau-Luen Tham
ABSTRACT
Recent advances in artificial intelligence (AI) and computer vision have transformed automatic license plate recognition (ALPR) into an important application for intelligent transportation systems. However, existing algorithms are not directly applicable in the Internet of Things (IoT) environment due to the hardware constraints of processing power. In this paper, we propose a lightweight and accurate IoT-based ALPR solution using deep learning. First, a newly trained YOLOv4-tiny model based on Malaysian car plate is attained via transfer learning. Second, OpenVINO is adopted to optimize the trained model for faster inference time. Third, centroid tracking and geofencing are utilized to collect multiple image instances of the same car plate. Fourth, OpenCV image processing is invoked to segment the characters of each image instance before feeding them into the Tesseract optical character recognition (OCR) engine for character recognition. Fifth, a weighted selection algorithm is designed to choose the best car plate number among the pooled samples. Lastly, the entire solution is deployed in the Up Squared board and powered by the popular IoT Node-Red. Results reveal that the proposed solution has a frame per second (FPS) of 2.6 using Intel Movidius Myriad X, detection accuracy of 99.02 %, and license plate optical character recognition (OCR) accuracy of 78.23%.
- F. Zhu, Y. Lv, Y. Chen, X. Wang, G. Xiong and F. -Y. Wang, "Parallel Transportation Systems: Toward IoT-Enabled Smart Urban Traffic Control and Management," IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 10, pp. 4063-4071, Oct. 2020.Google Scholar
Cross Ref
- W. Riaz, A. Azeem, G. Chenqiang, Z. Yuxi, Saifullah and W. Khalid, "YOLO Based Recognition Method for Automatic License Plate Recognition," 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), Aug. 2020, pp. 87-90.Google Scholar
- C. Zhang, P. Patras and H. Haddadi, "Deep Learning in Mobile and Wireless Networking: A Survey," in IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2224-2287, Mar. 2019.Google ScholarCross Ref
- J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 779-788.Google ScholarCross Ref
- W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” European Conference on Computer Vision, vol. 9905, pp. 21–37, Oct. 2016.Google ScholarCross Ref
- J. -a. Kim, J. -Y. Sung and S. -h. Park, "Comparison of Faster-RCNN, YOLO, and SSD for Real-Time Vehicle Type Recognition," 2020 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia), Nov. 2020, pp. 1-4.Google Scholar
- C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proc. IEEE Conf.Comput. Vis. Pattern Recognit., Jul. 2016, pp. 2818–2826.Google ScholarCross Ref
- M. -L. Tham, J. L. Foo, Y. C. Chang, E. Morris, and N. Ramli, “Performance Study of Deep-Learning Based Surveillance Systems.” International Journal of Advanced Science and Technology, vol. 29, no. 1, pp. 206-212, Jan. 2020.Google Scholar
- OpenVINO Toolkit n.d., accessed 1 August 2021, <https://software.intel.com/en-us/openvinotoolkit>.Google Scholar
- C.K. Soon, K.C. Lin, C.Y. Jeng, and S.A. Suandi, "Malaysian Car Number Plate Detection and Recognition System," Australian Journal of Basic and Applied Sciences, vol. 6, no. 3, pp. 49-59, Mac. 2012.Google Scholar
- K. Yogheedha, A.S.A. Nasir, H. Jaafar, and S.M. Mamduh, “Automatic Vehicle License Plate Recognition System Based on Image Processing and Template Matching Approach,” 2018 International Conference on Computational Approach in Smart Systems Design and Applications (ICASSDA), Aug. 2018, pp. 1-8.Google ScholarCross Ref
- M. Abdullah, M.A.H. Bakhtan, and S.A. Mokhtar, “Number Plate Recognition of Malaysia Vehicles using Smearing Algorithm,” Science International, vol. 29, no. 4, pp. 823-827, Aug. 2017.Google Scholar
- N.L. Yaacob, A.A. Alkahtani, F.M. Noman, A.W.M. Zuhdi, and D. Habeeb, “License plate recognition for campus auto-gate system,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 21, no. 1, pp. 128-136. Jan. 2021.Google ScholarCross Ref
- S.A.G. Fakhar, M.H. Saad, A.K. Fauzan, R.H. Affendi, and M.A. Aidil, “Development of portable automatic number plate recognition (ANPR) system on Raspberry Pi,” International Journal of Electrical and Computer Engineering (IJECE), vol. 9, no. 3, pp. 1805-1813. Jun. 2019.Google ScholarCross Ref
- L.T.A. Al-Mahbashi, N.A.T. Yusof, S. Shaharum, M.S.A. Karim, and Faudzi, A.A.M, “Development of Automated Gate Using Automatic License Plate Recognition System,” Proceedings of the 10th National Technical Seminar on Underwater System Technology 2018, vol. 538, pp. 459-466, Feb. 2019.Google ScholarCross Ref
- Z. Selmi, M. Halima, and A. Alimi, “Deep learning system for automatic license plate detection and recognition,” 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 1132–1138, Nov. 2017.Google ScholarCross Ref
- S. Montazzolli, and C. Jung, “Real-time Brazilian license plate detection and recognition using deep convolutional neural networks,” 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 55–62, Oct. 2017.Google ScholarCross Ref
- C. -Z. Riel, Y. R. Juan, and S. -B. Ko, “License Plate Segmentation and Recognition System using Deep Learning and OpenVINO,” IET Intelligent Transport Systems, vol. 14, Jan. 2020.Google Scholar
- Chevry, J E n.d., PP Welcome to the License Plates Portal, accessed 29 August 2021, < http://plates.portal.free.fr/>.Google Scholar
- Alexey Bochkovskiy. Darknet: Open Source Neural Networks in Python. 2020. Available online: https://github.com/AlexeyAB/darknet (accessed on 21 August 2021).Google Scholar
- Tesseract OCR, accessed 21 August 2021, <https://github.com/tesseract-ocr/tesserac>Google Scholar
- Paultan n.d., New vs Old Gallery, <https://paultan.org/topics/new-vs-old-gallery/>.Google Scholar
- Low, D 2019, LPR Ranger Barrier System @ Malaysia, accessed 21 August 2021, <https://www.youtube.com/watch?v=oJ1sAD7IoNs&t=1s>.Google Scholar
- TechplanetTV 2021, Anti-clone Long-range Barrier Gate System in Malaysia, accessed 21 August 2021, <https://www.youtube.com/watch?v=lP11lVjU-84>.Google Scholar
- LILIN CCTV MALAYSIA 2019, LILIN ANPR Solution in Malaysia, accessed 21 August 2021, https://www.youtube.com/watch?v=Eb9E_ms0T9s.Google Scholar
- Intelligent Security Systems 2009, License Plate Recognition Technologies, accessed 21 August 2021, <https://www.youtube.com/watch?v=eaLlQhVAtz4&t=16s>.Google Scholar
Recommendations
Deep automatic license plate recognition system
ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image ProcessingAutomatic License Plate Recognition (ALPR) has important applications in traffic surveillance. It is a challenging problem especially in countries like in India where the license plates have varying sizes, number of lines, fonts etc. The difficulty is ...
Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning
AbstractAutomatic License Plate Recognition (ALPR) is an important research topic in the intelligent transportation system and image recognition fields. In this work, we address the problem of car license plate detection using a You Only Look ...
Research on License Plate Recognition Based on Deep Learning in Complex Scenarios
ICMLSC '23: Proceedings of the 2023 7th International Conference on Machine Learning and Soft ComputingThe license plate angle is unfixed, the vehicle position is ununiform, and the picture is not sufficiently illuminated which leads to the decrease of license plate recognition accuracy. In order to improve the accuracy of license plate recognition, a ...
Comments