Skip to main content
Springer Nature Link
Log in

On fault diagnosis using image-based deep learning networks based on vibration signals

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the development of complex and intelligent mechanical products, the reliability of rotating machinery equipment increasingly gains much attention. As essential parts of rotating machinery equipment, bearing failures are becoming one of the leading root causes of mechanical failures. The critical problem of the bearing fault diagnosis is the method based on mechanical vibration signals. However, an accurate fault diagnosis is constantly challenged due to the nonlinearity and non-stationarity of vibration signals. This work investigates the fault diagnosis of bearing based on vibration signals, and an image-based deep learning method is proposed, in which vibration signals in the time domain are transformed into intrinsic mode functions (IMFs), and uncorrelated IMFs are removed. Rest IMFs reconstruct signals, which are transferred into symmetrized dot pattern (SDP) images. The fault diagnosis is then formulated as an image classification problem and solved with deep learning networks. In this work, the parameters of SDP are selected by optimizing an image similarity to improve the classification accuracy. The presented method is demonstrated with an experimental fault diagnosis of rolling bearings, and results show that the accuracy of the proposed method can be up to 99%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data Availability

Data available on request from the authors.

References

  1. Ahmed HOA, Nandi AK (2021) Connected components-based colour image representations of vibrations for a two-stage fault diagnosis of roller bearings using convolutional neural networks. Chin J Mech Eng 34:1–21

    Article  Google Scholar 

  2. Akhenia P, Bhavsar K, Panchal J, Vakharia V (2021) Fault severity classification of ball bearing using SinGAN and deep convolutional neural network. Proc Inst Mech Eng C J Mech Eng Sci 236:3864-3877

  3. Aljemely AH, Xuan J, Xu L, Jawad FKJ, Al-Azzawi O (2021) Wise-local response convolutional neural network based on Naïve Bayes theorem for rotating machinery fault classification. Appl Intell 51:6932–6950

    Article  Google Scholar 

  4. Amarouayache IIE, Saadi MN, Guersi N, Boutasseta N (2020) Bearing fault diagnostics using EEMD processing and convolutional neural network methods. Int J Adv Manuf Technol 107:4077–4095

    Article  Google Scholar 

  5. Bai R, Zhang W, Zhao DH (2019) Fault diagnosis algorithm of wind power gearbox based on fuzzy neural network. Paper presented at the 2019 Chinese Control And Decision Conference (CCDC), Nanchang, China, 2019, pp. 3291-3296, https://doi.org/10.1109/CCDC.2019.8833026

  6. Bai RX, Xu QS, Meng Z, Cao LX, Xing KS, Fan FJ (2021) Rolling bearing fault diagnosis based on multi-channel convolution neural network and multiscale clipping fusion data augmentation. Measurement 184:17. https://doi.org/10.1016/j.measurement.2021.109885

    Article  Google Scholar 

  7. Benmoussa S, Djeziri MA, Sanchez R (2020) Support vector machine classification of currentdata for fault diagnosis and similarity-basedapproach for failure prognosis in wind turbinesystems. Artificial Intelligence Techniques for a Scalable Energy Transition. Springer, Cham. https://doi.org/10.1007/978-3-030-42726-9_7

  8. Cen J, Chen Z, Wu Y, Yang Z (2023) Intelligent machine fault diagnosis based on deep transfer convolutional neural network and extreme learning machine. Proc Inst Mech Eng, Part C: J Mech Eng Sci 237(9):2201–2212. https://doi.org/10.1177/09544062221136490

  9. Choudhary A, Mian T, Fatima S (2021) Convolutional neural network based bearing fault diagnosis of rotating machine using thermal images. Measurement 176:109196

    Article  Google Scholar 

  10. Dibaj A, Ettefagh MM, Hassannejad R, Ehghaghi MB (2021) A hybrid fine-tuned VMD and CNN scheme for untrained compound fault diagnosis of rotating machinery with unequal-severity faults. Expert Syst Appl 167:16. https://doi.org/10.1016/j.eswa.2020.114094

    Article  Google Scholar 

  11. Fen W, Min L, Gang W, Xu J, Ren B, Wang GL (2016) Fault diagnosis approach of gearbox based on Support Vector Machine with improved bi-layers quantum genetic optimization. Paper presented at the 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI)

  12. Fu Q, Jing B, He P, Si S, Wang Y (2018) Fault feature selection and diagnosis of rolling bearings based on EEMD and optimized Elman_AdaBoost algorithm. IEEE Sens J 18:5024–5034

    Article  Google Scholar 

  13. Gai J, Hu Y (2018) Research on fault diagnosis based on singular value decomposition and fuzzy neural network. Shock Vibration 2018; 1–7. https://doi.org/10.1155/2018/8218657

  14. Gu YK, Zeng L, Qiu GQ (2020) Bearing fault diagnosis with varying conditions using angular domain resampling technology, SDP and DCNN. Measurement 156. https://doi.org/10.1016/j.measurement.2020.107616

  15. Han S, Oh S, Jeong J (2021) Bearing fault diagnosis based on multiscale convolutional neural network using data augmentation. J Sensors 2021:6699637:6699631-6699637:6699614

    Article  Google Scholar 

  16. Hao SJ, Ge FX, Li YM, Jiang JY (2020) Multisensor bearing fault diagnosis based on one-dimensional convolutional long short-term memory networks. Measurement 159:8. https://doi.org/10.1016/j.measurement.2020.107802

    Article  Google Scholar 

  17. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. Paper presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778, https://doi.org/10.1109/CVPR.2016.90

  18. Hu MT, Wang GF, Ma KL, Cao ZH, Yang S (2021) Bearing performance degradation assessment based on optimized EWT and CNN. Measurement 172:13. https://doi.org/10.1016/j.measurement.2020.108868

    Article  Google Scholar 

  19. Huang G, Liu Z, Weinberger KQ (2016) Densely connected convolutional networks. Paper presented at the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243

  20. Huang NE (2000) New method for nonlinear and non-stationary time series analysis: empirical mode decomposition and Hilbert spectral analysis. Proceedings of SPIE - The International Society for Optical Engineering

  21. Huang WY, Cheng JS, Yang Y, Guo GY (2019) An improved deep convolutional neural network with multiscale information for bearing fault diagnosis. Neurocomputing 359:77–92. https://doi.org/10.1016/j.neucom.2019.05.052

    Article  Google Scholar 

  22. Iqbal M, Madan AK (2022) CNC machine-bearing fault detection based on convolutional neural network using vibration and acoustic signal. J Vib Eng Technol 10:1613–1621

    Article  Google Scholar 

  23. Khan MA, Kim YH, Choo J (2020) Intelligent fault detection using raw vibration signals via dilated convolutional neural networks. J Supercomputing 76(10):8086–8100. https://doi.org/10.1007/s11227-018-2711-0

    Article  Google Scholar 

  24. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680

    Article  MathSciNet  Google Scholar 

  25. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90

    Article  Google Scholar 

  26. Lessmeier C, Kimotho JK, Zimmer D, Sextro W (2016) Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A Benchmark Data Set for Data-Driven Classification

  27. Li H, Wang W, Huang P, Li Q (2020) Fault diagnosis of rolling bearing using symmetrized dot pattern and density-based clustering. Measurement 152:107293

    Article  Google Scholar 

  28. Li J, Yao X, Wang X, Yu Q, Zhang Y (2020) Multiscale local features learning based on BP neural network for rolling bearing intelligent fault diagnosis. Measurement 153:107419

    Article  Google Scholar 

  29. Li Y, Xu M, Zhao HY, Huang W (2016) Hierarchical fuzzy entropy and improved support vector machine based binary tree approach for rolling bearing fault diagnosis. Mech Mach Theory 98:114–132

    Article  Google Scholar 

  30. Liu J, Yu X (2021) Rolling element bearing fault diagnosis for complex equipment based on MFMD and BP neural network. J Phys: Conf Ser 1948

  31. Long Z, Zhang XF, He M, Huang SD, Qin GJ, Song DAY, ... Shao HD (2022) Motor fault diagnosis based on scale invariant image features. IEEE Trans Ind Inform 18(3):1605–1617. https://doi.org/10.1109/tii.2021.3084615

  32. Long Z, Zhang XF, Song DY, Tang Y, Huang SD, Liang WZ (2021) Motor fault diagnosis using image visual information and bag of words model. IEEE Sens J 21(19):21798–21807. https://doi.org/10.1109/jsen.2021.3102019

    Article  Google Scholar 

  33. Pang B, Liang J, Liu H, Dong J, Xu Z, Zhao X (2022) Intelligent bearing fault diagnosis based on multivariate symmetrized dot pattern and LEG transformer. Machines 10(7):550. https://doi.org/10.3390/machines10070550

  34. Plakias S, Boutalis YS (2020) Fault detection and identification of rolling element bearings with Attentive Dense CNN. Neurocomputing 405:208–217. https://doi.org/10.1016/j.neucom.2020.04.143

    Article  Google Scholar 

  35. Ren Z (2022) An optimized excitation signal design for identification of PWA model and application to automotive throttles. Measurement Control 56:844–856

    Article  Google Scholar 

  36. Ren Z, Kroll A, Sofsky M, Laubenstein F (2013) On physical and data-driven modeling of systems with friction: methods and application to automotive throttles. At-Automatisierungstechnik 61(3):155–171. https://doi.org/10.1524/auto.2013.0023

    Article  Google Scholar 

  37. Ren ZX, Ji XX (2023) On prediction of air pollutants with Takagi-Sugeno models based on a hierarchical clustering identification method. Atmos Pollut Res 14(4). https://doi.org/10.1016/j.apr.2023.101731

  38. Sharma K, Goyal D, Kanda R (2022) Intelligent fault diagnosis of bearings based on convolutional neural network using infrared thermography. Proc Inst Mech Eng, Part J: J Eng Tribol 236:2439-2446

  39. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Int Conf Learn Represent. (2015) https://doi.org/10.48550/arXiv.1409.1556

  40. Sun Y, Li S-Y, Wang X (2021) Bearing fault diagnosis based on EMD and improved Chebyshev distance in SDP image. Measurement 176:109100

    Article  Google Scholar 

  41. Sun YJ, Li SH (2022) Bearing fault diagnosis based on optimal convolution neural network. Measurement 190:10. https://doi.org/10.1016/j.measurement.2022.110702

    Article  Google Scholar 

  42. Sun YJ, Li SH, Wang YL, Wang XH (2021) Fault diagnosis of rolling bearing based on empirical mode decomposition and improved manhattan distance in symmetrized dot pattern image. Mech Syst Signal Process 159:23. https://doi.org/10.1016/j.ymssp.2021.107817

    Article  Google Scholar 

  43. Tang Y, Zhang XF, Qin GJ, Long Z, Huang SD, Song DAY, Shao HD (2022) Graph cardinality preserved attention network for fault diagnosis of induction motor under varying speed and load condition. IEEE Trans Industr Inf 18(6):3702–3712. https://doi.org/10.1109/tii.2021.3112696

    Article  Google Scholar 

  44. Touzout W, Benazzouz D, Gougam F, Afia A, Rahmoune C (2020) Hybridization of time synchronous averaging, singular value decomposition, and adaptive neuro fuzzy inference system for multi-fault bearing diagnosis. Adv Mech Eng 12(12). https://doi.org/10.1177/1687814020980569

  45. Wang H, Xu JW, Yan RQ, Gao RX (2020) A new intelligent bearing fault diagnosis method using SDP representation and SE-CNN. IEEE Trans Instrum Meas 69(5):2377–2389. https://doi.org/10.1109/tim.2019.2956332

    Article  Google Scholar 

  46. Wang L, Shao Y (2018) Crack fault classification for planetary gearbox based on feature selection technique and K-means clustering method. Chin J Mech Eng 31:1–11

    Article  Google Scholar 

  47. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  48. Xie Z, Yu D, Zhan C, Zhao Q, Wang J, Liu J, Liu J (2023) Ball screw fault diagnosis based on continuous wavelet transform and two-dimensional convolution neural network. Measurement Control 56:518–528

    Article  Google Scholar 

  49. Yang H, Li X, Zhang W (2021). Interpretability of deep convolutional neural networks on rolling bearing fault diagnosis. Meas Sci Technol. https://doi.org/10.1088/1361-6501/ac41a5

  50. Yu H, Li H-r, Li Y (2020) Vibration signal fusion using improved empirical wavelet transform and variance contribution rate for weak fault detection of hydraulic pumps. ISA Trans 107:385–401. https://doi.org/10.1016/j.isatra.2020.07.0

  51. Zhao BX, Cheng CM, Peng ZK, Dong XJ, Meng G (2020) Detecting the early damages in structures with nonlinear output frequency response functions and the CNN-LSTM model. IEEE Trans Instrum Meas 69(12):9557–9567. https://doi.org/10.1109/tim.2020.3005113

    Article  Google Scholar 

  52. Zhao J, Yang SP, Li Q, Liu YQ, Gu XH, Liu WP (2021) A new bearing fault diagnosis method based on signal-to-image mapping and convolutional neural network. Measurement 176:15. https://doi.org/10.1016/j.measurement.2021.109088

    Article  Google Scholar 

  53. Zhao Q, Wang B, Zhou G, Zhang W, Guan X, Feng W (2016) An improved fault diagnosis approach based on support vector machine. IEEE Int Conf Prognostics Health Manag (ICPHM) 2016:1–7

    Google Scholar 

  54. Zhu X, Hou D, Zhou P, Han Z, Yuan Y, Zhou W, Yin Q (2019) Rotor fault diagnosis using a convolutional neural network with symmetrized dot pattern images. Measurement 138:526–535

    Article  Google Scholar 

  55. Zhu ZY, Peng GL, Chen YH, Gao HJ (2019) A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing 323:62–75. https://doi.org/10.1016/j.neucom.2018.09.050

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China (Grant No. 12204345), Natural Science Foundation of Shanxi Province, China (Grant No. 20210302123188), and Natural Science Foundation for Young Scientists of Shanxi Province, China (Grant No. 202103021223107). The authors would like to thank anonymous reviewers for carefully reading the paper.

Author information

Authors and Affiliations

  1. College of Data Science, Taiyuan University of Technology, Jinzhong, Shanxi, China

    Zhenxing Ren & Jianfeng Guo

Authors
  1. Zhenxing Ren
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jianfeng Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Zhenxing Ren: Project administration, Conceptualization, Methodology, Funding acquisition, Writing – original draft. Jianfeng Guo: Investigation, Data curation, Writing – review & editing.

Corresponding author

Correspondence to Zhenxing Ren.

Ethics declarations

Competing interest

The authors declare that they have no known competing financial interests or personal relationships that can have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, Z., Guo, J. On fault diagnosis using image-based deep learning networks based on vibration signals. Multimed Tools Appl 83, 44555–44580 (2024). https://doi.org/10.1007/s11042-023-17384-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17384-5

Keywords