Skip to main content
SpringerLink
Log in

Forward-looking omnidirectional infrared pedestrian detection for driver assistance

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To improve the intelligent driving abilities about the day-night situation awareness and large airspace real-time acquisition, we employ a forward-looking omnidirectional infrared (FLOIR) system and propose a FLOIR pedestrian detection strategy which consists of off-line training and on-line identification. Firstly, the basic scheme of proposed strategy is given generally, and the thoughts of off-line training and on-line identification are introduced briefly. Secondly, the principle of off-line training is illustrated in detail, in which the EHOG describer is created to extract the pedestrian feature and the FART neural network is modified to train the samples. Thirdly, the on-line identification principle is described, integrated with the proposed methods of ROI segmentation, distortion correction, feature extraction and matching. Finally, to verify the adaptability of FLOIR pedestrian detection relative to the climates, the summer and winter experiments, the contrast experiment with deep learning method and the multi-scale pedestrian detection experiment are carried out. The results show that: the proposed strategy has better robustness and detection effect than the traditional methods of FART, ARTMAP and deep learning, and the all-weather accuracy of infrared pedestrian detection is more than 83%. In the next work, the infrared pedestrian database will be improved to further increase the detection accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Data sharing not applicable to this paper as no datasets were generated or analyzed during the current study.

References

  1. Arifin AZ, Asano A (2006) Image segmentation by histogram thresholding using hierarchical cluster analysis. Pattern Recogn Lett 27(13):1515–1521. https://doi.org/10.1016/j.patrec.2006.02.022

    Article  Google Scholar 

  2. Bai X, Wang Y, Liu H, Guo S (2018) Symmetry information based fuzzy clustering for infrared pedestrian segmentation. IEEE T Fuzzy Syst 26(4):1946–1959. https://doi.org/10.1109/TFUZZ.2017.2756827

    Article  Google Scholar 

  3. Biswas SK, Milanfar P (2017) Linear support tensor machine: pedestrian detection in thermal infrared images. IEEE T Image Process 26(9):4229–4242. https://doi.org/10.1109/TIP.2017.2705426

    Article  MathSciNet  MATH  Google Scholar 

  4. Brunet D, Vrscay ER, Wang Z (2012) On the mathematical properties of the structural similarity index. IEEE T Image Process 21(4):1488–1499. https://doi.org/10.1109/TIP.2011.2173206

    Article  MathSciNet  MATH  Google Scholar 

  5. Cai L, Zhu J, Zeng H, Chen J, Cai C, Ma KK (2018) HOG-assisted deep feature learning for pedestrian gender recognition. J Franklin I 355(4):1991–2008. https://doi.org/10.1016/j.jfranklin.2017.09.003

    Article  Google Scholar 

  6. Chen Z, Zhang J, Wen S, Li Y, Hong Q (2021) Competitive neural network circuit based on winner-take-all mechanism and online Hebbian learning rule. IEEE T VLSI Syst 29:1095–1107. https://doi.org/10.1109/TVLSI.2021.3069221

    Article  Google Scholar 

  7. Chen Y, Zhang G, Ma Y, Kang JU, Kwan C (2022) Small infrared target detection based on fast adaptive masking and scaling with iterative segmentation. IEEE Geosci Remote S 19:1–5. https://doi.org/10.1109/LGRS.2020.3047524

    Article  Google Scholar 

  8. Dai X, Duan Y, Hu J, Liu S, Hu C, He Y, Chen D, Luo C, Meng J (2018) Near infrared nighttime road pedestrians recognition based on convolutional neural network. Infrared Phys Technol 97:25–32. https://doi.org/10.1016/j.infrared.2018.11.028

    Article  Google Scholar 

  9. Ding M, Chen WH, Cao YF (2022) ThermaliInfrared single-pedestrian tracking for advanced driver assistance system. IEEE Trans Intell Veh 8(1):814–824. https://doi.org/10.1109/TIV.2022.3140344

    Article  Google Scholar 

  10. Faecett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874. https://doi.org/10.1016/j.patrec.2005.10.010

    Article  Google Scholar 

  11. Ganguly B, Dey D, Munshi S (2022) An unsupervised learning approach for road anomaly segmentation using RGB-D sensor for advanced driver assistance system. IEEE T on Intell Transp 23(10):19042–19053. https://doi.org/10.1109/TITS.2022.3164847

    Article  Google Scholar 

  12. Hao S, He T, Ma X, Yang L, Sun S (2022) Cross-scale infrared pedestrian detection based on dynamic feature optimization mechanism. Opt Precis Eng 30(9):2390–2403. https://doi.org/10.0492/4X.2022.19-2390-14

    Article  Google Scholar 

  13. Helali A, Ameur H, Górriz M, Ramírez J, Maaref H (2020) Hardware implementation of real-time pedestrian detection system. Neural Comput & Applic 32:12859–12871. https://doi.org/10.1007/s00521-020-04731-y

    Article  Google Scholar 

  14. Huang F, Shen X, Wang Q, Zhou B, Hu W, Shen H, Li L (2013) Correction method for fisheye image based on the virtual small-field camera. Opt Lett 38(9):1392–1394. https://doi.org/10.1364/ol.38.001392

    Article  Google Scholar 

  15. Huang C, Li X, Wen Y (2021) An OTSU image segmentation based on fruitfly optimization algorithm. Alex Eng J 60(1):183–188. https://doi.org/10.1016/j.aej.2020.06.054

    Article  Google Scholar 

  16. Khan MA，Mittal M, Goyal LM, Roy S (2021) A deep survey on supervised learning based human detection and activity classification methods. Multimed Tools Appl, 80: 27867–27923. https://doi.org/10.1007/s11042-021-10811-5

    Article  Google Scholar 

  17. Kim T, Kim S (2018) Pedestrian detection at night time in FIR domain: comprehensive study about temperature and brightness and new benchmark. Pattern Recogn 79:44–54. https://doi.org/10.1016/j.patcog.2018.01.029

    Article  Google Scholar 

  18. Kumar A, Banno A, Ono S, Oishi T, Ikeuchi K (2013) Global coordinate adjustment of the 3D survey models under unstable GPS condition. Seisan Kenkyu 65:91–95. https://doi.org/10.11188/SEISANKENKYU.65.91

    Article  Google Scholar 

  19. Lee YS, Chan YM, Fu LC, Hsiao PY (2015) Near-infrared-based nighttime pedestrian detection using grouped part models. IEEE T on Intell Transport Syst 16(4):1929–1940. https://doi.org/10.1109/TITS.2014.2385707

    Article  Google Scholar 

  20. Li L, Zhou F, Bai X (2018) Infrared pedestrian segmentation through background likelihood and object-biased saliency. IEEE T on Intell Transport Syst 19(9):2826–2844. https://doi.org/10.1109/TITS.2017.2761901

    Article  Google Scholar 

  21. Li C, Song D, Tong R, Tang M (2019) Illumination-aware faster R-CNN for robust multispectral pedestrian detection. Pattern Recogn 85:161–171. https://doi.org/10.1016/j.patcog.2018.08.005

    Article  Google Scholar 

  22. Lin YC, Chiang PY, Miaou S (2022) Enhancing deep-learning object detection performance based on fusion of infrared and visible images in advanced driver assistance systems. IEEE Access 10:105214–105231. https://doi.org/10.1109/ACCESS.2022.3211267

    Article  Google Scholar 

  23. Liu Q, Zhuang J, Kong S (2013) Detection of pedestrians for far-infrared automotive night vision systems using learning-based method and head validation. Meas Sci Technol 24:074022-1–074022-6. https://doi.org/10.1088/0957-0233/24/7/074022

    Article  Google Scholar 

  24. Lu Y, Zhang C, Tian Y, Guo J, Sun J (2022) An overview of deep learning based pedestrian detection algorithms. J Image Graph 24:2094–2111. https://doi.org/10.11834/jig.200831

    Article  Google Scholar 

  25. Ma M (2020) RETRACTED ARTICLE: infrared pedestrian detection algorithm based on multimedia image recombination and matrix restoration. Multimed Tools Appl 79:9267–9282. https://doi.org/10.1007/s11042-019-7444-4

    Article  Google Scholar 

  26. Manda MP, Park C, Oh B, Hyun D, Kim HS (2020) Pedestrian detection in infrared thermal images based on raised cosine distribution. 2020 International SoC design conference (ISOCC), Yeosu, Korea (South), pp. 278–279. https://doi.org/10.1109/ISOCC50952.2020.9332804

  27. Qi B, John V, Liu Z, Mita S (2016) Pedestrian detection from thermal images: a sparse representation based approach. Infrared Phys Technol 76:157–167. https://doi.org/10.1016/j.infrared.2016.02.004

    Article  Google Scholar 

  28. Wang W (2021) Infrared pedestrian detection method based on attention model. Pattern Recogn Image Anal 31:821–829. https://doi.org/10.1134/S1054661821040271

    Article  Google Scholar 

  29. Wang Y, Bai X (2018) Intensity inhomogeneity suppressed fuzzy C-means for infrared pedestrian segmentation. IEEE T on Intell Transport Syst 20(9):3361–3374. https://doi.org/10.1109/TITS.2018.2875159

    Article  Google Scholar 

  30. Wang X, Huang K, Knoll A (2019) Performance optimization of parallelized ADAS applications in FPGA-GPU heterogeneous systems: a case study with lane detection. IEEE Trans Intell Veh 4:519–531. https://doi.org/10.1109/TIV.2019.2938092

    Article  Google Scholar 

  31. Wang Z, Feng J, Zhang Y (2022) Pedestrian detection in infrared image based on depth transfer learning. Multimed Tools Appl 81:39655–39674. https://doi.org/10.1007/s11042-022-13058-w

    Article  Google Scholar 

  32. Wu D, Yuan C (2022) Threshold image segmentation based on improved sparrow search algorithm. Multimed Tools Appl 81:33513–33546. https://doi.org/10.1007/s11042-022-13073-x

    Article  Google Scholar 

  33. Yu X, Xiong S, He Y, Wong WE, Zhao (2016) Research on campus traffic congestion detection using BP neural network and Markov model. J of Inf Secur Appl 31(1): 54–60. https://doi.org/10.1016/j.jisa.2016.08.003

  34. Zhang J, Liu C, Wang B, Chen C, He J, Zhou Y, Li J (2022) An infrared pedestrian detection method based on segmentation and domain adaptation learning. Comput Electr Eng 99:107781. https://doi.org/10.1016/j.compeleceng.2022.107781

    Article  Google Scholar 

  35. Zhao Y, Yuan Z, Chen B (2020) Training cascade compact CNN with region-IoU for accurate pedestrian detection. IEEE T on Intell Transport Syst 21(9):3777–3787. https://doi.org/10.1109/TITS.2019.2933581

    Article  Google Scholar 

  36. Zhu Y, Yang J, Deng X, Xiao C, An W (2020) Infrared pedestrian detection based on attention mechanism. J Phys Conf Ser 1634:012032. https://doi.org/10.1088/1742-6596/1634/1/012032

    Article  Google Scholar 

Download references

Funding

This research was funded by the National Natural Science Foundation of China (grant number 62171467) and the Natural Science Foundation of Hebei Province (grant number F2021506004).

Author information

Authors and Affiliations

  1. Shijiazhuang campus, Army Engineering University of PLA, Shijiazhuang, 050003, China

    Jianjun Zhang, Fuyu Huang, Jing Hao & Yudan Chen

  2. The 54th research Institute of China Electronics Technology Group Corporation, Shijiazhuang, 050081, China

    Jianjun Zhang

  3. Chinese People’s Armed Police Force Research Institute, Beijing, 100081, China

    Yichao Chen

  4. Hebei University of Economics and Business, Shijiazhuang, 050003, China

    Jing Hao

Authors
  1. Jianjun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuyu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yichao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yudan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuyu Huang.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Huang, F., Chen, Y. et al. Forward-looking omnidirectional infrared pedestrian detection for driver assistance. Multimed Tools Appl 82, 45389–45410 (2023). https://doi.org/10.1007/s11042-023-15466-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15466-y

Keywords

Search

Navigation