Skip to main content
SpringerLink
Log in

Enhanced detection and recognition system for vehicles and drivers using multi-scale retinex guided filter and machine learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Accurate vehicle detection plays a vital role in intelligent transportation systems. Various day conditions, for instance, dawn, morning, noon, or non-uniform illuminations put restrictions on camera’s visibility. Such scenarios impact the performance of detection and recognition algorithms that are used in surveillance systems and autonomous driving. This paper aims to solve the aforementioned issues using machine learning methods, such as face detection and recognition. The core theme of this paper is the development of a vehicle detection and driver recognition system, which also focuses the situation where an input image is degraded by non-uniform illuminations. The proposed system is composed of four main processing modules: (i) image acquisition, (ii) image enhancement, (iii) object detection that locates vehicles’ and drivers’ faces, and (iv) the Pool of Face Recognition Algorithms (PoFRA), which uses four face recognition algorithms to conclude the driver’s identity. We implement suitable algorithms for each of the above-described modules to appraise its practicability. The system can be adjusted to work in different types of extreme weather conditions, such as strong or dim light. Experimental results demonstrate that the proposed system has significant potential to take the research on automated car parking systems to the next level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Some data can be shared on request.

Notes

  1. https://dragon.larc.nasa.gov/retinex/

  2. htttp://www.dlr.de/vabene

References

  1. Afifi M, Brown M (2019) “What else can fool deep learning? Addressing color constancy errors on deep neural network performance,” In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 243–252

  2. Afifi M, Derpanis K, Ommer B, Brown M (2021) “Learning multi-scale photo exposure correction,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9157–9167

  3. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns, application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Google Scholar 

  4. Ameen M, Eleyan A, Eleyan G (2017) Wavelet transform based face recognition using SURF descriptors. Int J Electron Electric Eng 5(1):94–98

    Google Scholar 

  5. Arash A, Ali A (2018) Predicting the future location of cars on urban street network by chaining spatial web services. IET Intell Transp Syst 12(8):793–800

    Google Scholar 

  6. Asadianfam S, Shamsi M, Kenari A (2021) TVD-MRDL: traffic violation detection system using MapReduce-based deep learning for large-scale data. Multimed Tools Appl 80(2):2489–2516

    Google Scholar 

  7. Badii C, Nesi P, Paoli I (2018) Predicting available parking slots on critical and regular services by exploiting a range of open data. IEEE Access 6:44059–44071

    Google Scholar 

  8. Bernardo B, Cirillo S, Cuofano M, Desiato D (2022) Enhancing spatial perception through sound: mapping human movements into MIDI. Multimed Tools Appl:1–22. https://doi.org/10.1007/s11042-021-11077-7

  9. Burgstahler D, Knapp F, Zöller S (2014) “Where is that car parked? A wireless sensor network-based approach to detect car positions,” 9th International Workshop on Practical Issues in Building Sensor Network Applications, Edmonton, Canada, pp. 514–522

  10. Caruccio L, Cirillo S, Deufemia V, Polese G (2021) “Efficient Validation of Functional Dependencies during Incremental Discovery,” In Proceedings of the 29th Italian Symposium on Advanced Database Systems, Pizzo Calabro, Italy, pp. 5–9

  11. Chao D, Loy C, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. Eur Conf Comput Vis:184–199

  12. Chen G, Wang F, Qu S, Chen K, Yu J (2020) Pseudo-image and sparse points: vehicle detection with 2D LiDAR revisited by deep learning-based methods. IEEE Trans Intell Transp Syst 22(12):7699–7711

    Google Scholar 

  13. Chil G, Musolino G, Polimeni A et al (2016) Transport models and intelligent transportation system to support urban evacuation planning process. IET Intell Transp Syst 10(4):279–286

    Google Scholar 

  14. Christopher R, Fabio G, James W et al (2018) City-wide emissions modelling using fleet probe vehicles. IET Intell Transp Syst 12(9):1181–1188

    Google Scholar 

  15. Chu W, Yao L, Cai D (2018) Multi-task vehicle detection with region-of-interest voting. IEEE Trans Image Process 27(1):432–441

    MathSciNet  Google Scholar 

  16. Daniel C, David S, Jose M et al (2018) Distribution of road hazard warning messages to distant vehicles in intelligent transport systems. IEEE Trans Intell Transp Syst 9(4):1152–1165

    Google Scholar 

  17. Deng J, Guo J, Xue N, Zafeiriou S (2019) “Arcface: additive angular margin loss for deep face recognition,” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690–4699

  18. Ding C, Tao D (2015) Robust face recognition via multimodal deep face representation. IEEE Trans Multimed 17(11):2049–2058

    Google Scholar 

  19. Farid A, Hussain F, Khan K, Shahzad M, Khan U, Mahmood Z (2023) A fast and accurate real-time vehicle detection method using deep learning for unconstrained environments. Appl Sci 30(5):1–30

    Google Scholar 

  20. Ghimire D, Lee J (2011) Nonlinear transfer function-based local approach for color image enhancement. IEEE Trans Consum Electron 57(2):858–865

    Google Scholar 

  21. Guo C, Li C, Guo J, Loy C, Hou J, Kwong S, Cong R (2020) “Zero-reference deep curve estimation for low-light image enhancement,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789

  22. Haitham C, You C, Philip A et al (2017) Blockchain-based dynamic key management for heterogeneous intelligent transportation systems. IEEE Internet Things J 4(6):1832–1843

    Google Scholar 

  23. Han W, Chang S, Liu D, Yu M, Witbrock M, Huang TS (2018)“Image super-resolution via dual-state recurrent networks,” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1654–1663

  24. Haq M, Shahzad A, Mahmood Z, Shah A, Muhammad N, Akram T (2019) Boosting the face recognition performance of ensemble based LDA for pose, non-uniform illuminations, and low-resolution images. KSII Trans Int Inf Syst 13(6):3144–3164

    Google Scholar 

  25. Hassaballah M, Mahmoud M, Kenk KM, Minaee S (2020) Vehicle detection and tracking in adverse weather using a deep learning framework. IEEE Trans Intell Transp Syst 22(7):4230–4242

    Google Scholar 

  26. Hu Y, He H, Xu C, Wang B, Lin S (2018) Exposure: a white-box photo post-processing framework. ACM Trans Graph 37(2):1–17

    Google Scholar 

  27. Hussain F, Afifi M, Moussa G (2018) A comprehensive study of the effect of spatial resolution and color of digital images on vehicle classification. IEEE Trans Intell Transp Syst 20(3):1181–1190

    Google Scholar 

  28. Jamil K, Joao B, Lucio F et al (2018) A2CPS: a vehicle-centric safety conceptual framework for autonomous transport systems. IEEE Trans Intell Transp Syst 19(6):1925–1939

    Google Scholar 

  29. Jiao Z, Xu B (2009) “An image enhancement approach using retinex and YIQ,” in International Conference on Information Technology and Computer Science (ITCS), pp. 476–479

  30. Kafai M, Bhanu B (2012) Dynamic bayesian networks for vehicle classification in video. IEEE Trans Industrial Inf 8(1):100–109

    Google Scholar 

  31. Lai W, Huang J, Ahuja N, Yang M (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition:624–632

  32. Leitloff J, Rosenbaum D, Kurz F (2014) An operational system for estimating road traffic information from aerial images. Remote Sens 6(11):11 315–11 111341

    Google Scholar 

  33. Liu K, Mattyus G (2015) Fast multiclass vehicle detection on aerial images. IEEE Geosci Remote Sens Lett 12(9):1938–1942

    Google Scholar 

  34. Liu L, Jiabao W, Zheng Z, Su H (2021) An improved approach for mining association rules in parallel using spark streaming. Int J Circ Theory Appl 49(4):1028–1039

    Google Scholar 

  35. Lu J, Plataniotis K, Venetsanopoulos A, Li S (2006) Ensemble-based discriminant learning with boosting for face recognition. IEEE Trans Neural Netw 17(1):166–178

    Google Scholar 

  36. Lv T, Zhang Y, Luo L, Gao X (2022) MAFFNet: real-time multi-level attention feature fusion network with RGB-D semantic segmentation for autonomous driving. Appl Opt 61(9):2219–2229

    Google Scholar 

  37. Maeda S (2020) “Unpaired image super-resolution using pseudo-supervision,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 291–300

  38. Mahmood Z, Ali T, Khattak S, Khan SU (2014) “A comparative study of baseline algorithms of face recognition,” 12th International Conference on Frontier of Information Technology (FIT), pp. 263–268

  39. Mahmood Z, Khan M, Jawad M, Khan SU, Yang LT (2015) “A parallel framework for object detection and recognition for secure vehicle parking,” 17th International Conference on High Performance Computing and Communications (HPCC), pp. 892–895

  40. Mahmood Z, Ali T, Khattak S, Hassan L, Khan SU (2015) Automatic player detection and identification for sports entertainment applications. Pattern Anal Applic 18(4):971–982

    MathSciNet  Google Scholar 

  41. Mahmood Z, Ali T, Khan SU (2016) Effects of pose and image resolution on automatic face recognition. IET Biometrics 5(2):111–119

    Google Scholar 

  42. Mahmood Z, Ali T, Muhammad N, Bibi N, Shahzad I, Azmat S (2017) EAR: enhanced augmented reality system for sports entertainment applications. KSII Trans Int Inf Syst 11(12):6069–6091

    Google Scholar 

  43. Mahmood Z, Muhammad N, Bibi N, Ali T (2017) A review on state-of-the-art face recognition approaches. Fractals 25(2):1750025-1–1750025-19

    Google Scholar 

  44. Mahmood Z, Muhammad N, Bibi N, Malik Y, Ahmad N (2018) Visual enhancement of human observatory system using multi-scale retinex. Inf Med Unlocked 13:9–20

    Google Scholar 

  45. Mahmood Z, Haneef O, Muhammad N, Khattak S (2018) Towards a fully automated car parking system. IET Intell Transp Syst 13(2):293–302

    Google Scholar 

  46. Mahmood Z, Bibi N, Usman M, Khan U, Muhammad N (2019) Mobile cloud based framework for sports applications. Multidim Syst Sign Process 30(4):1991–2019

    Google Scholar 

  47. Mahmood Z, Khan K, Khan U, Adil SH, Ali SSA, Shahzad M (2022) Towards automatic license plate detection. Sensors 22(3):1–19

    Google Scholar 

  48. Masood S, Syed MI, Khan K, Fayyaz A, Sultan F, Mahmood Z (2020) “A Robust Method to Enhance Degraded Color Images Under Diverse Condition,” 14th International Conference on Open Source Systems and Technologies (ICOSST), pp. 1–6

  49. Meng J, Cao T, Peng J, Wang Z, Wang S (2022) Polarized image near-natural color fusion algorithm for target detection. Appl Opt 61(6):1323–1330

    Google Scholar 

  50. Min K, Lee G, Lee S (2021) “ACNet: Mask-Aware Attention with Dynamic Context Enhancement for Robust Acne Detection,” In International Conference on Systems, Man, and Cybernetics (SMC), pp. 2724–2729

  51. Moranduzzo T, Melgani (2014) Automatic car counting method for unmanned aerial vehicle images. IEEE Trans Geosci Remote Sens 52(3):1635–1647

    Google Scholar 

  52. Reilly V, Idrees H, Shah M (2010) “Detection and tracking of large number of targets in wide area surveillance,” In Proceedings of the European Conference on Computer Vision, pp. 186–199

  53. Richardos D, Bellotti F, Bakas I et al (2018) Gamified flexible transportation service for on-demand public transport. IEEE Trans Intell Transp Syst 19(3):921–933

    Google Scholar 

  54. Sadia H, Azeem F, Ullah H, Mahmood Z, Khattak S, Khan G. Z (2018) “Color image enhancement using multi-scale retinex with guided filter,” 16th International Conference on Frontiers of Information Technology (FIT), pp. 82–87

  55. Silla A, Pirkko R, Lars L et al (2017) Quantifying the effectiveness of ITS in improving safety of VRUs. IET Intell Transp Syst 11(3):164–172

    Google Scholar 

  56. Sultan F, Khan K, Shah YA, Shahzad M, Khan U, Mahmood Z (2023) Towards automatic license plate recognition in challenging conditions. Appl Sci 13(6):1–30

    Google Scholar 

  57. Sun B, Tao W, Chen W (2008) “Luminance based MSR for color image enhancement,” in International Congress on Image and Signal Processing, pp. 358–362

  58. Szegedy C, Toshev A, Erhan D (2013) “Deep neural network for object detection,” in Adv Neural Inf Proces Syst, pp. 2553–2561

  59. Tuermer S, Kurz F, Reinartz P, Stilla U Airborne vehicle detection in dense urban areas using HoG features and disparity maps. IEEE J Sel Topics Appl Earth Observ Remote Sens 6(6, 2–13):2327–2337

  60. Viola P, Michael J (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Google Scholar 

  61. Wang Q, Guo G (2020) LS-CNN: characterizing local patches at multiple scales for face recognition. IEEE Trans Inf Forensics Secur 15:1640–1653

    Google Scholar 

  62. Wang Q, Guo G (2021) DSA-face: diverse and sparse attentions for face recognition robust to pose variation and occlusion. IEEE Trans Inf Forensics Secur 16:4534–4543

    Google Scholar 

  63. Wang W, Xi J, Liu C et al (2017) Human-centered feed-forward control of a vehicle steering system based on a driver's path-following characteristics. IEEE Trans Intell Transp Syst 18(6):1440–1453

    Google Scholar 

  64. Wang Z, Chen J, Hoi S (2020) Deep learning for image super-resolution: a survey. IEEE Trans Pattern Anal Mach Intell 43(10):3365–3387

    Google Scholar 

  65. Wang Q, Wu T, Zheng H, Guo G (2020) “Hierarchical pyramid diverse attention networks for face recognition,” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8326–8335

  66. Wen X, Zhao H, Wang N, Yuan H (2006) A rear-vehicle detection system for static images based on monocular vision. In: Proc. 9th Int. Conf. Control, autom., robot. Vis, pp 2421–2424

    Google Scholar 

  67. Yang J, Zhang D, Frangi A (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Google Scholar 

  68. Zaki P, William M, Soliman B, Alexsan K, Khalil K, Moursy M (2020) “Traffic signs detection and recognition system using deep learning. arXiv 2020, arXiv:2003.03256

  69. Zhan X, Qian X, Satish V (2016) A graph-based approach to measuring the efficiency of an urban taxi service system. IEEE Trans Intell Transp Syst 17(9):2479–2489

    Google Scholar 

  70. Zhang Z, Wang Z, Lin Z, Qi H (2019) “Image super-resolution by neural texture transfer,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7982–7991

  71. Zhang Z, Lu X, Cao G, Yang Y, Jiao L, Liu F (2021) ViT-YOLO: transformer-based YOLO for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision:2799–2808

  72. Zhang J, Liu T, Yin X, Wang X, Zhang K, Xu J, Wang D (2021) An improved parking space recognition algorithm based on panoramic vision. Multimed Tools Appl 80(12):18181–18209

    Google Scholar 

  73. Zhao J, Hao S, Dai C, Zhang H, Zhao L (2022) Improved vision-based vehicle detection and classification by optimized YOLOv4. IEEE Access 10:8590–8603

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, COMSATS University Islamabad, Abbottabad Campus, Pakistan

    Zahid Mahmood, Mohsin Shahzad, Ahmad Fayyaz & Uzair Khan

  2. Department of Avionics Engineering, Air University Islamabad, Islamabad, Pakistan

    Khurram Khan

  3. Faculty of Computer Science and Engineering, GIK Institute of Engineering Sciences and Technology, Topi, KPK, Pakistan

    Khurram Khan

Authors
  1. Zahid Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khurram Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohsin Shahzad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmad Fayyaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Uzair Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All Authors have equivalent contributions.

Corresponding authors

Correspondence to Zahid Mahmood or Khurram Khan.

Ethics declarations

Competing interests /competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahmood, Z., Khan, K., Shahzad, M. et al. Enhanced detection and recognition system for vehicles and drivers using multi-scale retinex guided filter and machine learning. Multimed Tools Appl 83, 15785–15824 (2024). https://doi.org/10.1007/s11042-023-16140-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16140-z

Keywords

Search

Navigation