Skip to main content
SpringerLink
Log in

A novel deep learning approach for intelligent fault diagnosis applications based on time-frequency images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep learning (DL) techniques have been gaining ground for intelligent equipment/process fault diagnosis applications. However, employing DL methods for such applications comes with its technical challenges. The DL methods are utilized to extract features from raw data automatically, which leads up to its own complications in data preprocessing and/or feature engineering phases. Moreover, another difficulty arises when DL methods are employed utilizing single type of sensor data as the performance of a fault diagnosis application is hindered. To address these issues, we propose utilization of a deep residual network-based multi-sensory data fusion method. The method is established on time-frequency images obtained by short-time Fourier transform to diagnose machine faults. The experimental results demonstrate that the proposed model combining different types of measured signals can diagnose bearing conditions on machines more effectively compared to a single type of measured signal in terms of diagnostic accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and material

The data that supports the findings of this study is available at https://mb.uni-paderborn.de/en/kat/main-research/datacenter/bearing-datacenter/data-sets-and-download (accessed on November 2020). The source code that is typed for the proposed method and used in the experiments is available at https://github.com/ozggultekin/MultisensoryDataFusionWithSTFT.

References

  1. Carvalho TP, Soares FA, Vita R, Francisco RdP, Basto JP, Alcalá SG (2019) A systematic literature review of machine learning methods applied to predictive maintenance. Comput Ind Eng 137:106024. https://doi.org/10.1016/j.cie.2019.106024

    Article  Google Scholar 

  2. Kumar P, Hati AS (2020) Review on machine learning algorithm based fault detection in induction motors. Arch Comput Methods Eng. https://doi.org/10.1007/s11831-020-09446-w

    Article  Google Scholar 

  3. Lei Y, Yang B, Jiang X, Jia F, Li N, Nandi AK (2020) Applications of machine learning to machine fault diagnosis: a review and roadmap. Mech Syst Signal Process 138:106587. https://doi.org/10.1016/j.ymssp.2019.106587

    Article  Google Scholar 

  4. Hoang D-T, Kang H-J (2019) A survey on deep learning based bearing fault diagnosis. Neurocomputing 335:327–335. https://doi.org/10.1016/j.neucom.2018.06.078

    Article  Google Scholar 

  5. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507. https://doi.org/10.1126/science.1127647

    Article  MathSciNet  MATH  Google Scholar 

  6. Zhang W, Yang D, Wang H (2019) Data-driven methods for predictive maintenance of industrial equipment: a survey. IEEE Syst J 13(3):2213–2227. https://doi.org/10.1109/JSYST.2019.2905565

    Article  Google Scholar 

  7. Zhang W, Li X, Ding Q (2019) Deep residual learning-based fault diagnosis method for rotating machinery. ISA Trans 95:295–305. https://doi.org/10.1016/j.isatra.2018.12.025

    Article  Google Scholar 

  8. Ma S, Chu F, Han Q (2019) Deep residual learning with demodulated time-frequency features for fault diagnosis of planetary gearbox under nonstationary running conditions. Mech Syst Signal Process 127:190–201. https://doi.org/10.1016/j.ymssp.2019.02.055

    Article  Google Scholar 

  9. Zhao M, Kang M, Tang B, Pecht M (2017) Deep residual networks with dynamically weighted wavelet coefficients for fault diagnosis of planetary gearboxes. IEEE Trans Industr Electron 65(5):4290–4300. https://doi.org/10.1109/TIE.2017.2762639

    Article  Google Scholar 

  10. Pandhare V, Singh J, Lee J Convolutional neural network based rolling-element bearing fault diagnosis for naturally occurring and progressing defects using time-frequency domain features. In: 2019 Prognostics and System Health Management Conference (PHM-Paris), 2019. IEEE, pp. 320–326. Doi: https://doi.org/10.1109/PHM-Paris.2019.00061

  11. Wang L-H, Zhao X-P, Wu J-X, Xie Y-Y, Zhang Y-H (2017) Motor fault diagnosis based on short-time Fourier transform and convolutional neural network. Chin J Mech Eng 30(6):1357–1368. https://doi.org/10.1007/s10033-017-0190-5

    Article  Google Scholar 

  12. Zhang Y, Xing K, Bai R, Sun D, Meng Z (2020) An enhanced convolutional neural network for bearing fault diagnosis based on time–frequency image. Measurement. https://doi.org/10.1016/j.measurement.2020.107667

    Article  Google Scholar 

  13. Qiao H, Wang T, Wang P, Qiao S, Zhang L (2018) A time-distributed spatiotemporal feature learning method for machine health monitoring with multi-sensor time series. Sensors 18(9):2932. https://doi.org/10.3390/s18092932

    Article  Google Scholar 

  14. Jing L, Wang T, Zhao M, Wang P (2017) An adaptive multi-sensor data fusion method based on deep convolutional neural networks for fault diagnosis of planetary gearbox. Sensors 17(2):414. https://doi.org/10.3390/s17020414

    Article  Google Scholar 

  15. Wei Y, Wu D, Terpenny J (2020) Robust incipient fault detection of complex systems using data fusion. IEEE Trans Instrum Meas 69(12):9526–9534. https://doi.org/10.1109/TIM.2020.3003359

    Article  Google Scholar 

  16. Xu X, Tao Z, Ming W, An Q, Chen M (2020) Intelligent monitoring and diagnostics using a novel integrated model based on deep learning and multi-sensor feature fusion. Measurement. https://doi.org/10.1016/j.measurement.2020.108086

    Article  Google Scholar 

  17. Varshney PK (1997) Multisensor data fusion. Electron Commun Eng J 9(6):245–253. https://doi.org/10.1049/ecej:19970602

    Article  Google Scholar 

  18. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778. Doi:https://doi.org/10.1109/CVPR.2016.90

  19. Lessmeier C, Kimotho J, Zimmer D, Sextro W (2019) KAt-DataCenter, chair of design and drive technology. Paderborn University, Paderborn

    Google Scholar 

  20. Lessmeier C, Kimotho JK, Zimmer D, Sextro W (2016) Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: a benchmark data set for data-driven classification. In: Proceedings of the European conference of the prognostics and health management society, pp. 05–08

  21. Meng Z, Guo X, Pan Z, Sun D, Liu S (2019) Data segmentation and augmentation methods based on raw data using deep neural networks approach for rotating machinery fault diagnosis. IEEE Access 7:79510–79522. https://doi.org/10.1109/ACCESS.2019.2923417

    Article  Google Scholar 

  22. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv: 1412.6980

  23. Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv preprint arXiv: 1605.07146

Download references

Acknowledgements

This research is supported in part by the Scientific and Technical Research Council of Turkey (TUBITAK) under 2232 International Fellowship for Outstanding Researchers Program with the grant number 118C252.

Author information

Authors and Affiliations

  1. Department of Informatics, Eskişehir Osmangazi University, Meşelik Campus, 26040, Eskişehir, Turkey

    Özgür Gültekin

  2. Department of Computer Engineering, Eskişehir Osmangazi University, Meşelik Campus, 26040, Eskişehir, Turkey

    Eyüp Çinar, Kemal Özkan & Ahmet Yazıcı

  3. Center of Intelligent Systems Application Research, Eskisehir Osmangazi University, Meşelik Campus, 26040, Eskişehir, Turkey

    Özgür Gültekin, Eyüp Çinar, Kemal Özkan & Ahmet Yazıcı

Authors
  1. Özgür Gültekin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eyüp Çinar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kemal Özkan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmet Yazıcı
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Eyüp Çinar and Kemal Özkan; Methodology: Kemal Özkan; Formal analysis and investigation: Özgür Gültekin and Eyüp Çinar; Writing—original draft preparation: Özgür Gültekin; Writing—review and editing: Eyüp Çinar and Kemal Özkan; Funding acquisition: Eyüp Çinar; Software: Özgür Gültekin; Resources: Ahmet Yazıcı; Supervision: Ahmet Yazıcı, Project administration: Eyüp Çinar.

Corresponding author

Correspondence to Eyüp Çinar.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gültekin, Ö., Çinar, E., Özkan, K. et al. A novel deep learning approach for intelligent fault diagnosis applications based on time-frequency images. Neural Comput & Applic 34, 4803–4812 (2022). https://doi.org/10.1007/s00521-021-06668-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06668-2

Keywords

Search

Navigation