Skip to main content
SpringerLink
Log in

Residual attention convolutional autoencoder for feature learning and fault detection in nonlinear industrial processes

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep learning has been successfully applied in process monitoring in recent years due to its powerful feature extraction. However, these monitoring methods are difficult to extract intrinsic representations of the process data in complex nonlinear processes. A new deep neural network, residual attention convolutional autoencoder (RACAE) is proposed for process monitoring. The unsupervised learning method of RACAE can extract representative features from high-dimensional data, which can significantly improve process monitoring performance in nonlinear processes. RACAE effectively integrates convolution calculation with an autoencoder to perform effective feature extraction of multivariate data. Moreover, residual attention block is embedded in the autoencoder to select these key features and then reduce the feature dimension for detector. A new process monitoring model is proposed and two kinds of statistics are developed for fault detection. The effectiveness of RACAE in fault detection is evaluated through a numerical case and two benchmark processes. The convolutional autoencoder based on residual attention provides a new approach for feature learning and process monitoring of complex processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Mishra DP (2018) Fault detection, location and classification of a transmission line. Neural Comput Appl 30(5):1377–1424

    Article  Google Scholar 

  2. Wan Z, Li J, Gao Y (2018) Monitoring and diagnosis process of abnormal consumption on smart power grid. Neural Comput Appl 30(1):21–28

    Article  Google Scholar 

  3. Ge Z (2017) Review on data-driven modeling and monitoring for plant wide industrial processes. Chemom Intell Lab Syst 171:16–25

    Article  Google Scholar 

  4. Liu X, Li K, Mcafee M (2012) Application of nonlinear PCA for fault detection in polymer extrusion processes. Neural Comput Appl 21(6):1141–1148

    Article  Google Scholar 

  5. Deng X, Tian X, Chen S (2018) Nonlinear process fault diagnosis based on serial principal component analysis. IEEE Trans Neural Netw Learn Syst 29(3):560–572

    Article  MathSciNet  Google Scholar 

  6. Zhong B, Wang J, Zhou J (2016) Quality-related statistical process monitoring method based on global and local partial least-squares projection. Ind Eng Chem Res 55(6):1609–1622

    Article  Google Scholar 

  7. Yu G (2015) Fault feature extraction using independent component analysis with reference and its application on fault diagnosis of rotating machinery. Neural Comput Appl 26(1):187–198

    Article  Google Scholar 

  8. Nor NM, Hussain MA, Hassan CRC (2019) Multi-scale kernel Fisher discriminant analysis with adaptive neuro-fuzzy inference system (ANFIS) in fault detection and diagnosis framework for chemical process systems. Neural Comput Appl 32(13):9283–9297

    Article  Google Scholar 

  9. Miao C, Lv Z (2020) Nonlinear chemical processes fault detection based on adaptive kernel principal component analysis. Syst Sci Control Eng Open Access J 8(1):350–358

    Article  Google Scholar 

  10. Kini KR, Madakyaru M (2019) Anomaly detection using multi-scale dynamic principal component analysis for Tennessee Eastman Process// 2019 Fifth Indian Control Conference (ICC)

  11. Yu J (2012) Local and global principal component analysis for process monitoring. J Process Control 22(7):1358–1373

    Article  Google Scholar 

  12. Wang Y, Sun F, Li B (2017) Multiscale neighborhood normalization-based multiple dynamic PCA monitoring method for batch processes with frequent operations. IEEE Trans Autom Sci Eng 15(3):1053–1064

    Article  Google Scholar 

  13. Wang Y, Sun F, Li X (2020) Compound dimensionality reduction based multi-dynamic kernel principal component analysis monitoring method for batch process with large-scale data sets. J Intell Fuzzy Syst 38(1):471–480

    Article  Google Scholar 

  14. Nagpal T, Yadwinder B, Brar S (2014) Artificial neural network approaches for fault classification: comparison and performance. Neural Comput Appl 25(7–8):1863–1870

    Article  Google Scholar 

  15. Ertunc HM, Ocak H, Aliustaoglu C (2013) ANN- and ANFIS-based multi-staged decision algorithm for the detection and diagnosis of bearing faults. Neural Comput Appl 22(1):435–446

    Article  Google Scholar 

  16. Ren L, Lv W, Jiang SW (2016) Fault diagnosis using a joint model based on sparse representation and SVM. IEEE Trans Instrum Meas 65(10):1–8

    Article  Google Scholar 

  17. Heydarzadeh M, Nourani M (2016) A two-stage fault detection and isolation platform for industrial systems using residual evaluation. IEEE Trans Instrum Meas 65(10):1–9

    Article  Google Scholar 

  18. Yoo YJ (2019) Fault detection method using multi-mode principal component analysis based on gaussian mixture model for sewage source heat pump system. Int J Control Autom Syst 17(8):2125–2134

    Article  Google Scholar 

  19. Simani S, Farsoni S, Castaldi P (2017) Data-driven techniques for the fault diagnosis of a wind turbine benchmark. Int J Appl Math Comput Sci 28(2):247–268

    Article  MathSciNet  Google Scholar 

  20. Chaouch H, Charfedine S, Ouni K (2017) Intelligent supervision approach based on multilayer neural PCA and nonlinear gain scheduling. Neural Comput Appl 31(4):1153–1163

    Article  Google Scholar 

  21. Li Y, Liu Y, Zhang C (2018) Discriminant diffusion maps based K-nearest-neighbour for batch process fault detection. Can J Chem Eng 96(2):484–496

    Article  Google Scholar 

  22. Hinton GE, Salakhutdinov RR (2016) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  23. Wang Y, Liu M, Bao Z (2018) Stacked sparse autoencoder with PCA and SVM for data-based line trip fault diagnosis in power systems. Neural Comput Appl 31:6719–6731

  24. Yu J, Zheng X, Wang S (2018) Stacked denoising autoencoder-based feature learning for out-of-control source recognition in multivariate manufacturing process. Qual Reliab Eng 35(3):204–223

    Google Scholar 

  25. Yu J, Zheng X (2019) Stacked convolutional sparse denoising auto-encoder for identification of defect patterns in semiconductor wafer map. Comput Ind 109:121–133

    Article  MathSciNet  Google Scholar 

  26. Yan S, Yan X (2019) Design teacher and supervised dual stacked autoencoders for quality-relevant fault detection in industrial process. Appl Soft Comput 81:105526

    Article  Google Scholar 

  27. Tang P, Peng KX, Zhang K, Chen ZW, Yang X, Li L (2018) A deep belief network-based fault detection method for nonlinear process. IFAC Papers OnLine 51(24):9–14

    Article  Google Scholar 

  28. Hu G, Li H, Xia Y, Luo L (2018) A deep Boltzmann machine and multi-grained scanning forest ensemble collaborative method and its application to industrial fault diagnosis. Comput Ind 100:287–296

    Article  Google Scholar 

  29. Yu J, Yan X (2018) Layer-by-Layer enhancement strategy of favorable features of the deep belief network for industrial process monitoring. Ind Eng Chem Res 57(45):15479–15490

    Google Scholar 

  30. Wang Y, Pan Z, Yuan X, Yang C, Gui W (2019) A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network. ISA Trans. https://doi.org/10.1016/j.isatra.2019.07.001

    Article  Google Scholar 

  31. Wang Y, Zhang D, Dai G (2020) classification of high-resolution satellite images using improved U-Net. Int J Appl Math Comput Sci 30(3):399–413

    MATH  Google Scholar 

  32. Lee KB, Cheon S, Kim CO (2017) A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes. IEEE Trans Semicond Manuf 30(2):135–142

    Article  Google Scholar 

  33. Wen L, Li X, Gao L (2019) A transfer convolutional neural network for fault diagnosis based on ResNet-50. Neural Comput Appl 22(1):435–446

    Google Scholar 

  34. Zhang H, Wang P, Gao X (2019) Amplitude-frequency images-based ConvNet: applications of fault detection and diagnosis in chemical processes. J Chemom 33(9):e3168

    Article  Google Scholar 

  35. Miyata S, Lim J, Akashi Y (2019) Fault detection and diagnosis for heat source system using convolutional neural network with imaged faulty behavior data. Sci Technol Built Environ 26:1–9

    Google Scholar 

  36. Masci J, Meier U, D Cireşan (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: International conference on artificial neural networks, ICANN 2011: Artificial Neural Networks and Machine Learning—ICANN 2011, 2011, pp 52–59

  37. Ji Y, Zhang H, Zhang Z (2021) CNN-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances. Inf Sci 546:835–857

    Article  MathSciNet  Google Scholar 

  38. Huang H, Hu X, Zhao Y, Makkie M, Dong Q, Zhao S, Guo L, Liu T (2017) Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans Med Imaging 37(7):1551–1561

    Article  Google Scholar 

  39. Mnih V, Heess N, Graves A (2014) Recurrent models of visual attention. In: Advances in neural information processing systems, pp 2204–2212

  40. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. Comput Sci. arXiv:1409.0473

  41. Lindsay G W (2015) Feature-based attention in convolutional neural networks. arXiv preprint arXiv

  42. He K, Zhang X, Ren S (2015) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  43. Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60(2):223–311

    Article  MathSciNet  Google Scholar 

  44. Shen X, Agrawal S (2006) Kernel density estimation for an anomaly based intrusion detection system, MLMTA, pp 161–167

  45. Mcavoy TJ, Ye N (1994) Base control for the Tennessee Eastman problem. Comput Chem Eng 18(5):383–413

    Article  Google Scholar 

  46. Johannesmeyer MC, Singhal A, Seborg DE (2002) Pattern matching in historical data. AICHE J 48(9):2022–2038

    Article  Google Scholar 

  47. Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  48. Zhang B, Li W, Hao J (2018) Adversarial adaptive 1-D convolutional neural networks for bearing fault diagnosis under varying working condition. arXiv

Download references

Acknowledgements

This research was supported by National Natural Science Foundation of China (No. 71777173).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Tongji University, Shanghai, 201804, China

    Xing Liu & Jianbo Yu

  2. Statistics Department, Rutgers, the State University of New Jersey, New Brunswick, NJ, 08854, USA

    Lyujiangnan Ye

Authors
  1. Xing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianbo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lyujiangnan Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbo Yu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Yu, J. & Ye, L. Residual attention convolutional autoencoder for feature learning and fault detection in nonlinear industrial processes. Neural Comput & Applic 33, 12737–12753 (2021). https://doi.org/10.1007/s00521-021-05919-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05919-6

Keywords

Search

Navigation