Abstract
The early fault diagnosis is a kind of important technology to ensure the normal and reliable operation of wind turbines. However, due to the potential presence of noisy labels in health condition dataset and the weakly explanation of the deep neural network decisions, the performance of fault diagnosis is severely limited. In this paper, a framework called normalized recurrent neural network (NRNN) is proposed for noisy label fault diagnosis, in which the normalized long short-term memory is used to improve the training process and the forward crossentropy loss is introduced to handle the negative effect of noisy labels. The effectiveness and superiority of the proposed framework are verified by four datasets with different noisy label proportions. Meanwhile, the layer-wise relevance propagation algorithm is applied to explore the decision of framework and by visualizing the relevances of input samples to framework decisions, the NRNN does not treat samples equally and prefers signal peaks for classification decisions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arjona-Medina, J.A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., & Hochreiter, S. (2018). Rudder: Return decomposition for delayed rewards. arXiv:1806.07857.
Arras, L., Montavon, G., Müller, K.R., & Samek, W. (2017). Explaining recurrent neural network predictions in sentiment analysis. arXiv:1706.07206.
Asr, M. Y., Ettefagh, M. M., Hassannejad, R., & Razavi, S. N. (2017). Diagnosis of combined faults in rotary machinery by non-naive bayesian approach. Mechanical Systems and Signal Processing, 85, 56–70. https://doi.org/10.1016/j.ymssp.2016.08.005.
Ba, J.L., Kiros, J.R., & Hinton, G.E. (2016). Layer normalization. arXiv:1607.06450.
Chen, J., Jing, H., Chang, Y., & Liu, Q. (2019). Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process. Reliability Engineering & System Safety, 185, 372–382. https://doi.org/10.1016/j.ress.2019.01.006.
Ghosh, A., Kumar, H., & Sastry, P. (2017). Robust loss functions under label noise for deep neural networks. In Thirty-First AAAI conference on artificial intelligence (pp. 1919–1925).
Goyal, D., Choudhary, A., Pabla, B., & Dhami, S. (2020). Support vector machines based non-contact fault diagnosis system for bearings. Journal of Intelligent Manufacturing, 31, 1275–1289. https://doi.org/10.1007/s10845-019-01511-x.
Hendrycks, D., Mazeika, M., Wilson, D., & Gimpel, K. (2018). Using trusted data to train deep networks on labels corrupted by severe noise. In Advances in Neural Information Processing Systems 31 (pp. 10456–10465). Curran Associates, Inc. http://papers.nips.cc/paper/8246-using-trusted-data-to-train-deep-networks-on-labels-corrupted-by-severe-noise.pdf
Hoang, D. T., & Kang, H. J. (2019). Rolling element bearing fault diagnosis using convolutional neural network and vibration image. Cognitive Systems Research, 53, 42–50. https://doi.org/10.1016/j.cogsys.2018.03.002.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
Jia, F., Lei, Y., Lu, N., & Xing, S. (2018). Deep normalized convolutional neural network for imbalanced fault classification of machinery and its understanding via visualization. Mechanical Systems and Signal Processing, 110, 349–367. https://doi.org/10.1016/j.ymssp.2018.03.025.
Jiao, J., Zhao, M., Lin, J., & Ding, C. (2019). Deep coupled dense convolutional network with complementary data for intelligent fault diagnosis. IEEE Transactions on Industrial Electronics, 66(12), 9858–9867. https://doi.org/10.1109/TIE.2019.2902817.
Lei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016). An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data. IEEE Transactions on Industrial Electronics, 63(5), 3137–3147. https://doi.org/10.1109/TIE.2016.2519325.
Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018). Machinery health prognostics: A systematic review from data acquisition to rul prediction. Mechanical Systems and Signal Processing, 104, 799–836. https://doi.org/10.1016/j.ymssp.2017.11.016.
Li, J., Li, X., He, D., & Qu, Y. (2020). Unsupervised rotating machinery fault diagnosis method based on integrated SAE-DBN and a binary processor. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-020-01543-8.
Li, X., Zhang, W., Ding, Q., & Sun, J. Q. (2019). Multi-layer domain adaptation method for rolling bearing fault diagnosis. Signal Processing, 157, 180–197. https://doi.org/10.1016/j.sigpro.2018.12.005.
Li, Y., Yang, Y., Li, G., Xu, M., & Huang, W. (2017). A fault diagnosis scheme for planetary gearboxes using modified multi-scale symbolic dynamic entropy and mrmr feature selection. Mechanical Systems and Signal Processing, 91, 295–312. https://doi.org/10.1016/j.ymssp.2016.12.040.
Liu, H., Zhou, J., Zheng, Y., Jiang, W., & Zhang, Y. (2018). Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders. ISA Transactions, 77, 167–178. https://doi.org/10.1016/j.isatra.2018.04.005.
Liu, T., & Tao, D. (2015). Classification with noisy labels by importance reweighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(3), 447–461. https://doi.org/10.1109/TPAMI.2015.2456899.
Lou, X., & Loparo, K. A. (2004). Bearing fault diagnosis based on wavelet transform and fuzzy inference. Mechanical Systems and Signal Processing, 18(5), 1077–1095. https://doi.org/10.1016/S0888-3270(03)00077-3.
Luo, J., Huang, J., & Li, H. (2020). A case study of conditional deep convolutional generative adversarial networks in machine fault diagnosis. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-020-01579-w.
Montavon, G., Samek, W., & Müller, K. R. (2018). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1–15. https://doi.org/10.1016/j.dsp.2017.10.011.
Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (pp. 807–814).
Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., & Qu, L. (2017). Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1944–1952).
Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., & Rabinovich, A. (2014). Training deep neural networks on noisy labels with bootstrapping. arXiv:1412.6596.
Sebastian, B., Alexander, B., Grégoire, M., Frederick, K., Klaus-Robert, M., Wojciech, S., et al. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10(7), e0130140. https://doi.org/10.1371/journal.pone.0130140.
Sturm, I., Lapuschkin, S., Samek, W., & Müller, K. R. (2016). Interpretable deep neural networks for single-trial eeg classification. Journal of Neuroscience Methods, 274, 141–145. https://doi.org/10.1016/j.jneumeth.2016.10.008.
Van, M., & Kang, H. J. (2015). Bearing defect classification based on individual wavelet local fisher discriminant analysis with particle swarm optimization. IEEE Transactions on Industrial Informatics, 12(1), 124–135. https://doi.org/10.1109/TII.2015.2500098.
Wang, C., Gan, M., et al. (2018a). Fault feature extraction of rolling element bearings based on wavelet packet transform and sparse representation theory. Journal of Intelligent Manufacturing, 29, 937–951. https://doi.org/10.1007/s10845-015-1153-2.
Wang, C., Gan, M., et al. (2019a). A supervised sparsity-based wavelet feature for bearing fault diagnosis. Journal of Intelligent Manufacturing, 30, 229–239. https://doi.org/10.1007/s10845-016-1243-9.
Wang, X., He, H., & Li, L. (2019b). A hierarchical deep domain adaptation approach for fault diagnosis of power plant thermal system. IEEE Transactions on Industrial Informatics, 15(9), 5139–5148. https://doi.org/10.1109/TII.2019.2899118.
Wang, Z. Y., Lu, C., & Zhou, B. (2018b). Fault diagnosis for rotary machinery with selective ensemble neural networks. Mechanical Systems and Signal Processing, 113, 112–130. https://doi.org/10.1016/j.ymssp.2017.03.051.
Wen, L., Li, X., Gao, L., & Zhang, Y. (2017). A new convolutional neural network-based data-driven fault diagnosis method. IEEE Transactions on Industrial Electronics, 65(7), 5990–5998. https://doi.org/10.1109/TIE.2017.2774777.
Wu, Q., Ding, K., & Huang, B. (2018). Approach for fault prognosis using recurrent neural network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1428-5.
Wu, X., Jiang, G., Wang, X., Xie, P., & Li, X. (2019). A multi-level-denoising autoencoder approach for wind turbine fault detection. IEEE Access, 7, 59376–59387. https://doi.org/10.1109/ACCESS.2019.2914731.
Xu, Q., Lu, S., Jia, W., & Jiang, C. (2019). Imbalanced fault diagnosis of rotating machinery via multi-domain feature extraction and cost-sensitive learning. Journal of Intelligent Manufacturing,. https://doi.org/10.1007/s10845-019-01522-8.
Yan, X., & Jia, M. (2018). A novel optimized svm classification algorithm with multi-domain feature and its application to fault diagnosis of rolling bearing. Neurocomputing, 313, 47–64. https://doi.org/10.1016/j.neucom.2018.05.002.
Yu, J. B. (2019). Evolutionary manifold regularized stacked denoising autoencoders for gearbox fault diagnosis. Knowledge-Based Systems, 178, 111–122. https://doi.org/10.1016/j.knosys.2019.04.022.
Zhang, Y., Li, X., Gao, L., & Li, P. (2018). A new subset based deep feature learning method for intelligent fault diagnosis of bearing. Expert Systems with Applications, 110, 125–142. https://doi.org/10.1016/j.eswa.2018.05.032.
Zhang, Z., Li, S., Wang, J., Xin, Y., & An, Z. (2019). General normalized sparse filtering: A novel unsupervised learning method for rotating machinery fault diagnosis. Mechanical Systems and Signal Processing, 124, 596–612. https://doi.org/10.1016/j.ymssp.2019.02.006.
Acknowledgements
The authors would like to thank the Key Research and Development Plan of Shanxi Province (Grant No. 201703D111027), Shanxi International Cooperation Project (Grant Nos. 201803D421039, 201903D421045), and the Foundation of Shanxi Key Laboratory of Advanced Control and Equipment Intelligence (Grant No. ACEI202001) for their support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Nie, X., Xie, G. A novel normalized recurrent neural network for fault diagnosis with noisy labels. J Intell Manuf 32, 1271–1288 (2021). https://doi.org/10.1007/s10845-020-01608-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-020-01608-8