Skip to main content
SpringerLink
Log in

Visualization enhancement of autonomous controlling vehicles system by thermal image processing technique

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The visual camera in the autonomous vehicles is inadequate to capture a prominent image of objects, especially in the night. The headlights of the car also have limited perceptibility up to few meters. So, it is utterly essential to implement such a technique so that visibility in the night would also be cleared as crystal. In this paper, a thermal imaging camera is proposed to be felicitated in autonomous vehicles for better identification of objects especially in the night when visibility is very less. This technique is framed to use as a data acquisition tool to recognize the objects. Due to the huge variation in grayscales and pseudo coloring values in the thermal image, a fuzzy-based CNN model is proposed to be applied to identify the boundaries of the objects. In this technique, the correlation between the thermal images of the moving object and its types is proposed to be trained with the novel FCNN model. The acquired data from different scenarios would also be compared with driving safety experts. By inter-connecting these practices, a novel and unique real-time thermographic image processing would be framed for the autonomous vehicle system. The proposed technique is implemented in a grayscaled thermal image to verify the process and its analysis indicates the robustness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Alam MS, Bognar JG, Hardie RC, Yasuda BJ (2000) Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames. IEEE Trans Instrum Measur 49(5):915–923

    Article  Google Scholar 

  2. Alexa P, Solař J, Čmiel F, Valíček P, Kadulová M (2018) Infrared thermographic measurement of the surface temperature and emissivity of glossy materials. J Build Phys 41(6):533–546

    Article  Google Scholar 

  3. Algarni AD (2020) Efficient object detection and classification of heat emitting objects from infrared images based on deep learning. Multimed Tools Appl 79(19):13403–13426

    Article  Google Scholar 

  4. Azari MN, Sedighi M, Mehdi S (2016) Intelligent fault detection in power distribution systems using thermos-grams by ensemble classifiers. Automatika 57(4):862–870

    Article  Google Scholar 

  5. Browne M, Ghidary SS, Mayer NM (2008) Convolutional neural networks for image processing with applications in mobile robotics. In: Speech, Audio, Image and Biomedical Signal Processing using Neural Networks. Springer, Berlin, pp 327–349

  6. Burigana L, Magnini L (2017) Image processing and analysis of radar and lidar data: new discoveries in Verona southern lowland (Italy). STAR: Sci Technol Archaeol Res 3(2):490–509

    Article  Google Scholar 

  7. Caillas C (1990) Thermal imaging for autonomous vehicle in outdoor scenes. In: EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications, pp 651–658. IEEE

  8. Chacón M, Mario I (2006) Fuzzy logic for image processing: definition and applications of a fuzzy image processing scheme. Advanced Fuzzy Logic Technologies in Industrial Applications, pp 101–113

  9. de MMG Bittencourt T, Gonzaga A (2012) Digital image processing techniques for uncooled LWIR thermal camera. In: Electro-Optical and Infrared Systems: Technology and Applications IX, vol 8541. International Society for Optics and Photonics, pp 85410Z

  10. De Dick R, Duin RPW, Egmont-Petersen M, Van Vliet LJ, Verbeek PW (2003) Nonlinear image processing using artificial neural networks. Adv Imaging Electron Phys 126:351–450

    Article  Google Scholar 

  11. Giacomin J (2010) Thermal: seeing the world through 21st century eyes. Papadakis

  12. https://en.wikipedia.org/wiki/Black-body-radiation, on April 23, 2020

  13. https://en.wikipedia.org/wiki/Category-theory, on April 23, 2020

  14. https://en.wikipedia.org/wiki/Functor, on April 23, 2020

  15. https://en.wikipedia.org/wiki/Proportionality-(mathematics), on April 23, 2020

  16. https://en.wikipedia.org/wiki/Activation-function, on April 23, 2020

  17. Iakymchuk T, Rosado-Muñoz A, Guerrero-Martínez JF, Bataller-Mompeán M, Francés-Víllora JV (2015) Simplified spiking neural network architecture and STDP learning algorithm applied to image classification. EURASIP J Image Video Process 2015(1):1–11

    Article  Google Scholar 

  18. Lawrence M, Ashley SF, Lupton M, McEwen RK, Wilson M (2007) Signal processing core for high performance thermal imaging. In: Infrared Technology and Applications XXXIII, vol 6542. International Society for Optics and Photonics, pp 654215

  19. Li T, Wang Y, Chen Z, Wang R (2001) Linear feature extraction for infrared image. In: Image Extraction, Segmentation, and Recognition, vol 4550. International Society for Optics and Photonics, pp 281–286

  20. Liu C, Zhou C, Cao W, Li F, Jia P (2020) A novel design and implementation of autonomous robotic car based on ROS in indoor scenario. Robotics 9 (1):19

    Article  Google Scholar 

  21. Liu G, Liu Z, Liu S, Ma J, Wang F (2018) Registration of infrared and visible light image based on visual saliency and scale invariant feature transform. EURASIP J Image Video Process 2018(1):1–12

    Article  Google Scholar 

  22. LeBeau T (2019) Thermal imaging for safer autonomous vehicles. In: Infrared Technology and Applications XLV, vol 11002. International Society for Optics and Photonics, pp 110021H

  23. Mas JF, Flores JJ (2008) The application of artificial neural networks to the analysis of remotely sensed data. Int J Remote Sens 29(3):617–663

    Article  Google Scholar 

  24. Muhadi NA, Abdullah AF, Bejo SK, Mahadi MR, Mijic A (2020) Image segmentation methods for flood monitoring system. Water 12(6):1825

    Article  Google Scholar 

  25. Nath S, Agarwal S, Pandey GN (2015) Mathematical foundation based inter-connectivity modelling of thermal image processing technique for fire protection. EAI Endorsed Trans Creat Technol 5:2

    Google Scholar 

  26. Nam Y, Nam Y-C (2018) Vehicle classification based on images from visible light and thermal cameras. EURASIP J Image Video Process 2018(1):1–9

    Article  Google Scholar 

  27. Peterson BJ (2000) Infrared imaging video bolometer. Rev Sci Instrum 71(10):3696–3701

    Article  Google Scholar 

  28. Rossignoli I, Benito PJ, Herrero AJ (2015) Reliability of infrared thermography in skin temperature evaluation of wheelchair users. Spinal Cord 53(3):243–248

    Article  Google Scholar 

  29. Song E, Lee H, Choi J, Sangyoun L (2018) AHD: Thermal image-based adaptive hand detection for enhanced tracking system. IEEE access 6:12156–12166

    Article  Google Scholar 

  30. Tan S-T, Chen K, Ong S, Chew W (2008) Utilization of spectral vector properties in multivariate chemometrics analysis of hyperspectral infrared imaging data for cellular studies. Analyst 133(10):1395–1408

    Article  Google Scholar 

  31. Thakur R (2017) Infrared Sensors for Autonomous Vehicles. In: Recent Development in Optoelectronic Devices. IntechOpen

  32. Tsukamoto T, Tanaka S (2013) Patternable Temperature Sensitive Paint using Eu (TTA) 3 for the Micro Thermal Imaging. J Phys: Conf Ser 476(1):012073. IOP Publishing

  33. Winter J, Stein MA (1973) Computer image processing techniques for automated breast thermogram interpretation. Comput Biomed Res 6(6):522–529

    Article  Google Scholar 

  34. Zhang J, Jia X, Li J (2015) Integration of scanning and image processing algorithms for lane detection based on fuzzy method. J Intell Fuzzy Syst 29(6):2779–2786

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of CSE, NIT Trichy, Trichy, India

    Sayantan Nath & C. Mala

Authors
  1. Sayantan Nath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Mala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayantan Nath.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nath, S., Mala, C. Visualization enhancement of autonomous controlling vehicles system by thermal image processing technique. Multimed Tools Appl 81, 41035–41058 (2022). https://doi.org/10.1007/s11042-022-13077-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13077-7

Keywords

Search

Navigation