Skip to main content
SpringerLink
Log in

Edge AI System Using a Thermal Camera for Industrial Anomaly Detection

  • Conference paper
  • First Online:
Science and Technologies for Smart Cities (SmartCity 360 2021)

Included in the following conference series:

Abstract

Predictive maintenance plays an important role in reducing long-term maintenance costs, unplanned downtime, and improving the lifetime of industrial machines. A common trait of machines is that they produce heat while working, resulting in a temperature pattern. Temperature can be a key parameter for monitoring the condition of machines, further aiding the diagnostics of problems. This paper presents an Internet of Things (IoT) system that monitors and detects thermal anomalies in industrial machines using deep neural networks (DNNs). The proposed system enables the DNN to run and make predictions inside a microcontroller, reducing the amount of data that needs to be transmitted to any external server. Furthermore, this system uses a platform that centralizes multiple sensors with the option of communicating with a server that runs two additional neural networks that are specialized in highlighting zones of interest in the thermal image and monitoring the temperature behavior over time. The system was tested in a laboratory and two industrial environments. Overall, the system performed well and can detect machine anomalies while also drastically reducing the amount of data needed to be transmitted. The system also presented high adaptability to different environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/eloquentarduino/tinymlgen.

  2. 2.

    LSTM for temperature forecast.

  3. 3.

    LSTM for standard deviation forecast.

References

  1. Zhu, Z., et al.: Preventive maintenance subject to equipment unavailability. IEEE Trans. Reliab. (2019). https://doi.org/10.1109/tr.2019.2913331

    Article  Google Scholar 

  2. Bagavathiappan, S., Lahiri, B.B., Saravanan, T., Philip, J., Jayakumar, T.: Infrared thermography for condition monitoring - a review, Infrared Phys. Technol. 60, 35-55 (2013) https://doi.org/10.1016/j.infrared.2013.03.006

  3. Kanawaday, A., Sane, A.: Machine Learning for Predictive Maintenance of Industrial Machines using IoT Sensor Data (2018) https://doi.org/10.1109/ICSESS.2017.8342870

  4. Paolanti, M., et al.: Machine Learning approach for Predictive Maintenance in Industry 4.0. (2018) https://doi.org/10.1109/MESA.2018.8449150

  5. Sun, W., Liu, J., Yue, Y.: AI-enhanced offloading in edge computing: when machine learning meets industrial IoT. IEEE Netw. 33(5), 68–74 (2019). https://doi.org/10.1109/MNET.001.1800510

    Article  Google Scholar 

  6. Kammerer, K., et al.: Anomaly detections for manufacturing systems based on sensor data—insights into two challenging real-world production settings. Sensors (Switzerland) 19(24), 5370 (2019). https://doi.org/10.3390/s19245370

    Article  Google Scholar 

  7. Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., Zhang, J.: Edge intelligence: paving the last mile of artificial intelligence with edge computing. Proc. IEEE 107(8), 1738–1762 (2019). https://doi.org/10.1109/JPROC.2019.2918951

    Article  Google Scholar 

  8. Funk, F., Bucksch, T., Mueller-Gritschneder, D.: ML training on a tiny microcontroller for a self-adaptive neural network-based DC motor speed controller. In: Gama, J., et al. (eds.) ITEM/IoT Streams -2020. CCIS, vol. 1325, pp. 268–279. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66770-2_20

    Chapter  Google Scholar 

  9. Andrade, P., et al.: An Unsupervised TinyML Approach Applied for Pavement Anomalies Detection Under the Internet of Intelligent Vehicles (2021) https://doi.org/10.1109/MetroInd4.0IoT51437.2021.9488546

  10. Cerutti, G., Prasad, R., Farella, E.: Convolutional Neural Network on Embedded Platform for People Presence Detection in Low Resolution Thermal Images (2019) https://doi.org/10.1109/ICASSP.2019.8682998

  11. Kraft, M., Aszkowski, P., Pieczyński, D., Fularz, M.: Low-cost thermal camera-based counting occupancy meter facilitating energy saving in smart buildings. Energies 14(15), 4542 (2021). https://doi.org/10.3390/en14154542

    Article  Google Scholar 

  12. Spasov, G., et al.: Using IR Array MLX90640 to Build an IoT Solution for ALL and Security Smart Systems. (2019) https://doi.org/10.1109/ET.2019.8878637

  13. Maier, A., Sharp, A., Vagapov, Y.: Comparative analysis and practical implementation of the ESP32 microcontroller module for the internet of things. (2017) https://doi.org/10.1109/ITECHA.2017.8101926

  14. Masdani, M.V., Darlis, D.: A Comprehensive Study on MQTT as a Low Power Protocol for Internet of Things Application. (2018) https://doi.org/10.1088/1757-899X/434/1/012274

  15. Mikołajczyk, A., Grochowski, M.: Data augmentation for improving deep learning in image classification problem. (2018) https://doi.org/10.1109/IIPHDW.2018.8388338

  16. Dokic, K.: Microcontrollers on the edge – is ESP32 with camera ready for machine learning? In: El Moataz, A., Mammass, D., Mansouri, A., Nouboud, F. (eds.) ICISP 2020. LNCS, vol. 12119, pp. 213–220. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51935-3_23

    Chapter  Google Scholar 

  17. Malhotra, P., Vig, L., Shroff, G., Agarwal, P.: Long Short Term Memory Networks for Anomaly Detection in Time Series. (2015)

    Google Scholar 

  18. Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. (2016)

    Google Scholar 

  19. Chen, P., et al.: Time Series Forecasting of Temperatures using SARIMA: An Example from Nanjing. (2018) https://doi.org/10.1088/1757-899X/394/5/052024

  20. Ying, X.: An Overview of Overfitting and its Solutions. (2019) https://doi.org/10.1088/1742-6596/1168/2/022022

  21. Kristo, M., et al.: Thermal object detection in difficult weather conditions using YOLO. IEEE Access8, 125459-125476 (2020), https://doi.org/10.1109/ACCESS.2020.3007481

  22. Banuls, A., et al.: Object Detection from Thermal Infrared and Visible Light Cameras in Search and Rescue Scenes. (2020) https://doi.org/10.1109/SSRR50563.2020.9292593

  23. Adarsh, P., Rathi, P., Kumar, M.: YOLO v3-Tiny: Object Detection and Recognition using one stage improved model. (2020) https://doi.org/10.1109/ICACCS48705.2020.9074315

  24. Gholamy, A., Kreinovich, V., Kosheleva, O.: Why 70/30 or 80/20 relation between training and testing sets : a pedagogical explanation. Dep. Tech. Reports (2018)

    Google Scholar 

  25. Huda, A.S.N., Taib, S.: Application of infrared thermography for predictive/preventive maintenance of thermal defect in electrical equipment. Appl. Therm. Eng. 61(2), 220–227 (2013). https://doi.org/10.1016/j.applthermaleng.2013.07.028

    Article  Google Scholar 

Download references

Acknowledgment

This project was funded by national funds (PIDDAC), through the FCT – Fundação para a Ciência e Tecnologia and FCT/MCTES under the scope of the project UIDB/05549/2020 and through the special FCT program "Verão Com Ciência" with the project 2Ai Summer School (process 77, 20/229). Moreover, it was also funded by SmartHealth “NORTE-01–0145-FEDER-000045”, supported by Northern Portugal Regional Operational Programme (Norte2020), under the Portugal 2020 Partnership Agreement, through the European Regional Development Fund (FEDER).

Author information

Authors and Affiliations

  1. 2Ai – School of Technology, IPCA, Barcelos, Portugal

    Vítor M. Oliveira & António H. J. Moreira

Authors
  1. Vítor M. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. António H. J. Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to António H. J. Moreira .

Editor information

Editors and Affiliations

  1. Instituto Politécnico de Viana do Castelo, Viana do Castelo, Portugal

    Sara Paiva

  2. Auckland University of Technology, Auckland, New Zealand

    Xuejun Li

  3. Instituto Politécnico de Viana do Castelo, Viana do Castelo, Portugal

    Sérgio Ivan Lopes

  4. SRM Institute of Science and Technology, Chennai, India

    Nishu Gupta

  5. Howard University, Washington, DC, USA

    Danda B. Rawat

  6. Stoke-on-Trent, Birmingham, UK

    Asma Patel

  7. Politecnico di Milano, Milan, Italy

    Hamid Reza Karimi 

Appendix

Appendix

1.1 Temporal Anomaly Detection Results

Fig. 8.
figure 8

Temporal Anomaly Detection Model results. (a) Soldering Iron, (b) Hydraulic Refrigeration System and (c) 3-Phase Power Cable System.

Full size image

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Oliveira, V.M., Moreira, A.H.J. (2022). Edge AI System Using a Thermal Camera for Industrial Anomaly Detection. In: Paiva, S., et al. Science and Technologies for Smart Cities. SmartCity 360 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 442. Springer, Cham. https://doi.org/10.1007/978-3-031-06371-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06371-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06370-1

  • Online ISBN: 978-3-031-06371-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation