Skip to main content
SpringerLink
Log in

A fault diagnosis method for few-shot industrial processes based on semantic segmentation and hybrid domain transfer learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Fault diagnosis of industrial processes plays an important role in avoiding heavy losses and ensuring production safety. Complex industrial processes often have many working conditions, and the actual industrial process often concentrates on certain working conditions. As a result, the running time of some working conditions is shorter, so the data of these conditions are difficult to obtain. However, almost all fault diagnosis methods based on Deep Learning (DL) requires a large amount of data. Therefore, it is a big challenge to realize the fault diagnosis of few-shot industrial processes. In order to solve these problems, this paper proposes a Deep Feature Transfer Fusion (DFTF) framework based on hybrid domain transfer learning. The purpose is to take few-shot working conditions as the target domain and carry out fault diagnosis for them. As the features of industrial process images are more complex, this paper introduces Pyramid Hybrid Vision Transformer (PHVT) model, which have stronger feature extraction capabilities and spatial perception, as feature extraction module. In order to improve the transferability of the model, this paper introduces the In-Cross Domain Hybrid Transfer Learning (ICTL) method. By fusing the general object features from ResNet50 which pre-trained under public dataset of common object and the features extracted from PHVT which pre-trained under industrial dataset of multiple working conditions, the adaptability of the model to different scenes is enhanced. The experimental results based on Pronto dataset of Process System Engineering lab of Cranfield University show that the proposed transfer learning method has excellent performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are not publicly available due to other research being carried out, but are available from the corresponding author on reasonable request.

References

  1. Garcia-Alvarez D et al (2023) Integrating PCA and structural model decomposition to improve fault monitoring and diagnosis with varying operation points. Eng Appl Artif Intell 122:106145

    Article  Google Scholar 

  2. Huang X et al (2022) Fault diagnosis of automobile power seat with acoustic analysis and retrained SVM based on smartphone. Meas 202:111699

    Article  Google Scholar 

  3. Wang M, Xu X, Yan Z (2023) Online fault diagnosis of PV array considering label errors based on distributionally robust logistic regression. Renew Energy 203:68–80

    Article  Google Scholar 

  4. Souza RM et al (2021) Deep learning for diagnosis and classification of faults in industrial rotating machinery. Comput Ind Eng 153:107060

    Article  Google Scholar 

  5. Zhang J et al (2023) Gated recurrent unit-enhanced deep convolutional neural network for real-time industrial process fault diagnosis. Process Saf Environ Prot 175:129–149

    Article  Google Scholar 

  6. Vogado LHS et al (2018) Leukemia diagnosis in blood slides using transfer learning in CNNs and SVM for classification. Eng Appl Artif Intell 72:415–422

    Article  Google Scholar 

  7. Kwak M, Lee J (2023) Diagnosis-based domain-adaptive design using designable data augmentation and Bayesian transfer learning: target design estimation and validation. Appl Soft Comput 143:110459

    Article  Google Scholar 

  8. Mythili T, Anbarasi A (2022) A concatenation of deep and texture features for medicinal trash image classification using EnSegNet-DNN-based transfer learning. Mater Today: Proceed 62:4691–4698

    Google Scholar 

  9. Dosovitskiy A, Beyer L, Kolesnikov A et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. https://doi.org/10.48550/arXiv.2010.11929

  10. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  11. Deng J et al (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  12. Shi P, Gao H, Yu Y et al (2022) Intelligent fault diagnosis of rolling mills based on dual attention- guided deep learning method under imbalanced data conditions. Measurement 204:111993

  13. Chen J, Shang G (2023) Localization and imaging of internal hidden defects in concrete slabs based on deep learning of vibration signals. J Build Eng 76:107087

    Article  Google Scholar 

  14. Pang Y et al (2022) A deep learning based multiple signals fusion architecture for power system fault diagnosis. Sustain Energ Grids Netw 30:100660

    Article  Google Scholar 

  15. Wang J, Yang Y, Li N (2023) Randomization-based neural networks for image-based wind turbine fault diagnosis. Eng Appl Artif Intell 121:106028

    Article  Google Scholar 

  16. Shao H et al (2021) Intelligent fault diagnosis of rotor-bearing system under varying working conditions with modified transfer convolutional neural network and thermal images. IEEE Trans Indust Inf 17(5):3488–3496

    Article  Google Scholar 

  17. Attallah O, Ibrahim RA, Zakzouk NE (2022) Fault diagnosis for induction generator-based wind turbine using ensemble deep learning techniques. Energy Rep 8:12787–12798

    Article  Google Scholar 

  18. Wang W, Xie E, Li X et al (2021) Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 568–578

  19. Liu Z, Lin Y, Cao Y et al (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022

  20. Chu X et al (2021) Twins: revisiting the design of spatial attention in vision transformers. Adv Neural Inf Proces Syst 34:9355–9366

    Google Scholar 

  21. Yang Y et al (2023) MMViT-Seg: a lightweight transformer and CNN fusion network for COVID-19 segmentation. Comput Methods Prog Biomed 230:107348

    Article  Google Scholar 

  22. Hong-wei F, Ning-ge M, Xu-hui Z, Ce-yi X, Jia-teng M, Yang Y (2022) New intelligent fault diagnosis approach of rolling bearing based on improved vibration gray texture image and vision transformer. Proc Inst Mech Eng C: J Mech Eng Sci. https://doi.org/10.1177/09544062221085871

  23. Tang, X, Xu Z, Wang Z (2022) A novel fault diagnosis method of rolling bearing based on integrated vision transformer model. Sensors 22(10):3878

  24. Zabin M, Choi H-J, Uddin J (2022) Hybrid deep transfer learning architecture for industrial fault diagnosis using Hilbert transform and DCNNLSTM. J Supercomput 79:5181–5200. https://doi.org/10.1007/s11227-022-04830-8

    Article  Google Scholar 

  25. Li C et al (2023) Attention-based deep meta-transfer learning for few-shot fine-grained fault diagnosis. Knowl-Based Syst 264:110345

    Article  Google Scholar 

  26. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  27. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  28. Ronneberger, O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, pp 234–241

  29. Aslan MF (2022) A robust semantic lung segmentation study for CNN-based COVID-19 diagnosis. Chemom Intell Lab Syst 231:104695

    Article  Google Scholar 

  30. Liu X, Shen H, Gao L et al (2023) Lung parenchyma segmentation based on semantic data augmentation and boundary attention consistency. Biomed Signal Process Control 80:104205

  31. Wang K (2023) Contrastive learning-based semantic segmentation for in-situ stratified defect detection in additive manufacturing. J Manuf Syst 68:465–476

    Article  Google Scholar 

  32. Stief A et al (2019) A heterogeneous benchmark dataset for data analytics: multiphase flow facility case study. J Process Control 79:41–55

    Article  Google Scholar 

Download references

Funding

This work was supported by National Natural Science Foundation of China (62003215, 62003215, 22308217).

Author information

Authors and Affiliations

  1. Department of Control Science and Engineering, School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Ying Tian, Yiwei Wang & Wei Zhang

  2. Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, 200237, China

    Xin Peng

Authors
  1. Ying Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Tian.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tian, Y., Wang, Y., Peng, X. et al. A fault diagnosis method for few-shot industrial processes based on semantic segmentation and hybrid domain transfer learning. Appl Intell 53, 28268–28290 (2023). https://doi.org/10.1007/s10489-023-04979-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04979-6

Keywords

Search

Navigation