Skip to main content
SpringerLink
Log in

Using the deep neural networks for normal and abnormal situation recognition in the automatic access monitoring and control system of vehicles

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

A new mathematical model of the intelligent access monitoring and control system based on the cybernetic approach is proposed for solving the problems of vehicle access to the territory of an organization. The distinctive feature of the mathematical model is the ability to take into account and recognize normal and abnormal situations at the protected object and develop control actions. To localize vehicles and recognize their license plates, the composition of traditional methods of image processing and two-pass classification performed by the developed modified architecture of convolutional neural network MobileNet is offered. The composition allows to define additional identification features of the object. The proposed adaptations allow to recognize the situation in real time with low computational costs and high accuracy. The natural experiment has shown that the integration of the modern hardware means and algorithms of object detection and recognition, even in the rough conditions of street closed-circuit television monitoring, provides not less than 96% accuracy, and the processing time of one frame is not more than 0.094 s based on the Nvidia GeForce 1080Ti graphic processor. High recognition accuracy without loss of speed in the real-time mode is achieved by integrating the modern hardware means and the algorithms of object detection and recognition. The program module in Python using the Tensorflow and Keras library is developed for carrying out the access control functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aarathi KS, Abraham A (2017) Vehicle color recognition using deep learning for hazy images. In: ICICCT, pp 335–339

  2. Nsaif Al-Mayyahi M. Hazim N, Aboalmaaly M (2018) Vehicle detection and license plate recognition system. Int J Eng Technol(UAE). 7. 3170-3174. 10.14419/ijet.v7i4.19154. (2018)

  3. Boonsim N, Prakoonwit S (2016) Car make and model recognition under limited lighting conditions at night. Pattern Anal Appl 20:1915–1207

    Google Scholar 

  4. Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3642–3649

  5. Amosov OS, Ivanov YS, Zhiganov SV (2017) Human localization in the frame of the video stream using an algorithm based on growing neural gas and fuzzy inference. Comput Opt 41(1):46–58

    Article  Google Scholar 

  6. Soldatova OP, Garshin AA (2010) The application of a convolutional neural network for recognition of handwritten figures. Comput Opt 34(2):252–259

    Google Scholar 

  7. Lajish VL, Sunil KK (2015) Mobile phone based vehicle license plate recognition for road policing. arXiv preprint. arXiv:1504.01476

  8. Ullah F, Anwar H, Shahzadi I, Rehman A, Mehmood S, Niaz S, Awan KK, Ajmal Kwak Daehan (2019) Barrier access control using sensors platform and vehicle license plate characters recognition. Sensors. 19:3015. https://doi.org/10.3390/s19133015

    Article  Google Scholar 

  9. Hui L, Chunhua S (2016) Reading car license plates using deep convolutional neural networks and LSTMs. arXiv preprint arXiv:1601.05610

  10. Hao LY, Li J, Guo G (2019) A multi-target corner pooling-based neural network for vehicle detection. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04486-1

    Article  Google Scholar 

  11. Du X (2019) Complex environment image recognition algorithm based on GANs and transfer learning. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04018-x

    Article  Google Scholar 

  12. Masood SZ, Guang S, Dehghan A, Ortiz EG (2016) License plate detection and recognition using deeply learned convolutional neural networks. arXiv preprint arXiv:1703.07330

  13. Plate recognizer https://platerecognizer.com/ Accessed 30 Nov (2019)

  14. Automatic License Plate Recognition https://www.openalpr.com/index.html Accessed 30 Nov (2019)

  15. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: Efficient convolutional neural networks for mobile vision. arXiv preprint arXiv:1704.04861

  16. Verma A, Hebbalaguppe R, Vig L, Kumar S, Hassan E (2015) Pedestrian detection via mixture of CNN experts and thresholded aggregated channel features. In: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop. pp 555–563

  17. Amosov OS, Baena SG, Ivanov YS, Soe H (2017) Roadway gate automatic control system with the use of fuzzy inference and computer vision technologies. In: The 12th IEEE conference on industrial electronics and applications, pp 706–711

  18. Amosov OS, Amosova SG, Ivanov YS (2018) Intelligent access monitoring and control system of physical person. In: 2018 IEEE XXI international conference on soft computing and measurements (SCM), pp 352–355

  19. Amosov OS (2004) Neural network and fuzzy methods for estimating stochastic systems, p 352. Komsomolsk-on-Amur: Doctor’s thesis

  20. Binary classification performances measure cheat sheet https://kaggle2.blob.core.windows.net/ Accessed 20 Dec (2017)

  21. Redmon J, Divvala SK, Girshick RB, Farhadi A (2015) You only look once: unified, real-time object detection. arXiv preprint. arXiv:1506.02640v4

  22. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going deeper with convolutions. Technical Report. arXiv preprint. arXiv:1704.04861

  23. Intersection over Union (IoU) for object detection https://www.pyimagesearch.com/. Accessed 20 Nov (2017)

  24. Viola PJ, Snow D (2005) Detecting pedestrians using patterns of motion and appearance. Int J Comput Vis 63(2):153–161

    Article  Google Scholar 

  25. OpenCV (Open Source Computer Vision Library). https://github.com/opencv/opencv. Accessed 15 Sept (2017)

  26. Yadav G, Maheshwari S, Agarwal A (2014) Contrast limited adaptive histogram equalization based enhancement for real time video system. In: ICACCI, pp 2392–2397

  27. Matas J, Chum O, Urban M, Pajdla T (2002) Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of of British machine vision conference, pp 384–396

  28. Simonyan K, Zisserman A Very deep convolutional networks for large-scale image recognition. arXiv preprint. arxiv:1409.1556

  29. Keras: the Python Deep Learning library https://keras.io. Accessed 31 Sept (2017)

  30. Siam M, Gamal M, Abdel-Razek M, Yogamani S, Jagersand M (2018) RTSeg: Real-time semantic segmentation comparative study. arXiv preprint. arXiv:1803.02758

  31. ImageNet http://www.image-net.org/. Accessed 15 Dec (2017)

  32. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? arXiv preprint. arxiv:1411.1792

  33. Hinton G, Srivastava N, Swersky K (2017) Lecture 6a overview of mini-batch gradient descent. http://www.cs.toronto.edu/ (2012) Accessed 30 Sept

  34. AMCS Komsomolsk-na-Amure State University Access System https://github.com/IdentySergey/ncaa-app. Accessed 30 Nov (2019)

  35. AMCS Komsomolsk-na-Amure State University dataset http://evernow.ru/acs.zip. Accessed 30 Jan (2018)

  36. Tesseract OCR https://github.com/tesseract-ocr/tesseract. Accessed 10 Sept (2017)

Download references

Acknowledgements

The work was financially supported by the Russian Ministry of Education and Science—the Project 2.1898.2017/4.6 “Designing the Mathematical and Algorithmic Ware of Intelligent Information and Telecommunication System for Higher Educational Institution Security.”

Author information

Authors and Affiliations

  1. Laboratory of Intellectual Control Systems and Modelling, V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia, 117342

    Oleg Semenovich Amosov

  2. Laboratory of Cyber-Physical Systems, V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia, 117342

    Svetlana Gennadievna Amosova

  3. Department of Industrial Electronics, Komsomolsk-na-Amure State University, Komsomolsk-on-Amur, Russia, 681013

    Yuriy Sergeevich Ivanov

  4. Department of Network and System Administration, Komsomolsk-na-Amure State University, Komsomolsk-on-Amur, Russia, 681013

    Sergey Victorovich Zhiganov

Authors
  1. Oleg Semenovich Amosov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Svetlana Gennadievna Amosova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuriy Sergeevich Ivanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sergey Victorovich Zhiganov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oleg Semenovich Amosov.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amosov, O.S., Amosova, S.G., Ivanov, Y.S. et al. Using the deep neural networks for normal and abnormal situation recognition in the automatic access monitoring and control system of vehicles. Neural Comput & Applic 33, 3069–3083 (2021). https://doi.org/10.1007/s00521-020-05170-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05170-5

Keywords

Search

Navigation