Skip to main content
SpringerLink
Log in

CSiamese: a novel semi-supervised anomaly detection framework for gas turbines via reconstruction similarity

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The main problem in data-driven anomaly detection of gas turbines is that the monitoring data consists of only a very small number of abnormal samples with the overwhelming majority of normal samples. To address this problem, this paper develops a novel semi-supervised anomaly detection framework, namely CSiamese, and the parameters of the framework are optimized only using normal samples, with the ultimate purpose of improving the anomaly detection performance on imbalanced data sets. First, the convolutional auto-encoder is used to learn the reconstructed representation of the input sample. Second, the Siamese network is selected to learn to measure the similarity between the input and its reconstructed representation under noise conditions. Besides, a new loss function is developed by improving the contrastive loss, namely triangle loss, and can reduce the risk of collapsing solutions of the Siamese network when only using positive sample pairs. Third, maximum likelihood estimation is used to set the proper detection threshold to separate abnormal samples from normal samples. Finally, the effectiveness of the developed CSiamese has been evaluated using the real monitoring data of gas turbines and a public CIFAR-10 data set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tahan M, Tsoutsanis E, Muhammad M, Karim ZA (2017) Performance-based health monitoring, diagnostics and prognostics for condition-based maintenance of gas turbines: a review. Appl Energy 198:122–144

    Article  Google Scholar 

  2. Yan W, Yu L (2019) On accurate and reliable anomaly detection for gas turbine combustors: a deep learning approach. arXiv preprint arXiv:1908.09238

  3. Yazdani S, Montazeri-Gh M (2020) A novel gas turbine fault detection and identification strategy based on hybrid dimensionality reduction and uncertain rule-based fuzzy logic. Comput Ind 115:103–131

    Article  Google Scholar 

  4. Mousavi M, Chaibakhsh A, Jamali A, Kordestani M, Saif M (2022) A new fault diagnosis approach for heavy-duty gas turbines. IEEE/ASME Trans Mechatron

  5. Asgari H, Chen X, Sainudiin R (2013) Modelling and simulation of gas turbines. Int J Model Ident Control 20(3):253–270

    Article  Google Scholar 

  6. Simon DL, Rinehart AW (2014) A model-based anomaly detection approach for analyzing streaming aircraft engine measurement data, vol 45752. American Society of Mechanical Engineers

  7. Tian Z, Jin T, Wu B, Ding F (2011) Condition based maintenance optimization for wind power generation systems under continuous monitoring. Renew Energy 36(5):1502–1509

    Article  Google Scholar 

  8. Li X, Ding Q, Sun J (2018) Remaining useful life estimation in prognostics using deep convolution neural networks. Reliabil Eng Syst Saf 172:1–11

    Article  Google Scholar 

  9. Tang Z, Chen Z, Bao Y, Li H (2019) Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Health Monit 26(1):2296

    Article  Google Scholar 

  10. Garg S, Kaur K, Kumar N, Kaddoum G, Zomaya AY, Ranjan R (2019) A hybrid deep learning-based model for anomaly detection in cloud datacenter networks. IEEE Trans Netw Serv Manag 16(3):924–935

    Article  Google Scholar 

  11. Zhong S, Liu D, Lin L, Zhao M, Fu X, Guo F (2020) A novel anomaly detection method for gas turbines using weight agnostic neural network search. Asia-Pacific international symposium on advanced reliability and maintenance modeling, pp 1–6. IEEE

  12. Zhou X, Liang W, Shimizu S, Ma J, Jin Q (2020) Siamese neural network based few-shot learning for anomaly detection in industrial cyber-physical systems. IEEE Trans Ind Inf 17(8):5790–5798

    Article  Google Scholar 

  13. Meng L, Zhao M, Cui Z, Zhang X, Zhong S (2022) Empirical mode reconstruction: Preserving intrinsic components in data augmentation for intelligent fault diagnosis of civil aviation hydraulic pumps. Comput Ind 134:103557

    Article  Google Scholar 

  14. Yoo Y (2020) Data-driven fault detection process using correlation based clustering. Comput Ind 122:103–279

    Article  Google Scholar 

  15. Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: a survey. arXiv preprint arXiv:1901.03407

  16. Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016) High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recogn 58:121–134

    Article  Google Scholar 

  17. Zhu L, Borlaug B, Lin L, Holden J, Gonder J (2022) Light-duty vehicle trip classification using one-class novelty detection and exhaustive feature extraction. IEEE Trans Ind Appl

  18. Duarte DP, Nogueira RN, Bilro LB (2019) Semi-supervised gaussian and t-distribution hybrid mixture model for water leak detection. Meas Sci Technol 30(12):125109

    Article  Google Scholar 

  19. Hyunseong L, Hyung JL, Aditi C (2021) Data-driven system health monitoring technique using autoencoder for the safety management of commercial aircraft. Neural Comput Appl 33:3235–3250

    Article  Google Scholar 

  20. Yan W (2020) Detecting gas turbine combustor anomalies using semi-supervised anomaly detection with deep representation learning. Cogn Comput 12(2):398–411

    Article  Google Scholar 

  21. Ribeiro M, Lazzaretti AE, Lopes HS (2018) A study of deep convolutional auto-encoders for anomaly detection in videos. Pattern Recogn Lett 105:13–22

    Article  Google Scholar 

  22. Xuyun F, Hui L, Zhong S, Lin L (2019) Aircraft engine fault detection based on grouped convolutional denoising autoencoders. Chin J Aeronaut 32(2):296–307

    Article  Google Scholar 

  23. Turchenko V, Luczak A (2017) Creation of a deep convolutional auto-encoder in caffe. In: IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, vol 2. IEEE, pp 651–659

  24. Suganuma M, Ozay M, Okatani T (2018) Exploiting the potential of standard convolutional autoencoders for image restoration by evolutionary search. International Conference on Machine Learning. PMLR, pp 4771–4780

  25. Ren Y, Wang N, Li M, Xu Z (2020) Deep density-based image clustering. Knowl-Based Syst 197:105841

    Article  Google Scholar 

  26. Hamdi S, Snoussi H, Abid M (2020) Fine-tuning a pre-trained cae for deep one class anomaly detection in video footage. In: Mediterranean conference on pattern recognition and artificial intelligence. Springer, pp 3–17

  27. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, pp 1735–1742

  28. Wang Z, Bovik AC (2009) Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98–117

    Article  Google Scholar 

  29. Larsen ABL, Sonderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. International Conference on Machine Learning. PMLR, pp 1558–1566

  30. Zhou C, Paffenroth RC (2017) Anomaly detection with robust deep autoencoders. Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 665–674

  31. Sohangir S, Wang D (2017) Improved sqrt-cosine similarity measurement. J Big Data 4(1):1–13

    Article  Google Scholar 

  32. Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. IEEE computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp 539–546

  33. Liu D, Zhong S, Lin L, Zhao M, Xia X, Fu X, Cui Z (2021) A novel performance prediction method for gas turbines using the prophet model. In: 2021 IEEE international conference on sensing, diagnostics, prognostics, and control (SDPC). IEEE, pp 203–208

  34. Li Z, Zhong S, Lin L (2017) Novel gas turbine fault diagnosis method based on performance deviation model. J Propul Power 33(3):730–739

    Article  Google Scholar 

  35. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  36. Chow JK, Su Z, Wu J, Tan PS, Mao X, Wang Y-H (2020) Anomaly detection of defects on concrete structures with the convolutional autoencoder. Adv Eng Inform 45:101105

    Article  Google Scholar 

  37. Finke T, Kramer M, Morandini A, Muck A, Oleksiyuk I (2021) Autoencoders for unsupervised anomaly detection in high energy physics. J High Energy Phys 2021(6):1–32

    Article  Google Scholar 

  38. Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  39. Bromley J, Guyon I, LeCun Y, Sackinger E, Shah R (1993) Signature verification using a siamese time delay neural network. Adv Neural Inf Process Syst 6:737–744

    Google Scholar 

  40. Ahrabian K, BabaAli B (2019) Usage of autoencoders and siamese networks for online handwritten signature verification. Neural Comput Appl 31:9321–9334

    Article  Google Scholar 

  41. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Article  MathSciNet  Google Scholar 

  42. Compare M, Martini F, Mattafirri S, Carlevaro F, Zio E (2016) Semi-Markov model for the oxidation degradation mechanism in gas turbine nozzles. IEEE Trans Reliab 65(2):574–581

    Article  Google Scholar 

  43. Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. Asian Conference on Computer Vision. Springer, pp 622–637

  44. Ling CX, Huang J, Zhang H et al (2003) Auc: a statistically consistent and more discriminating measure than accuracy. Int Jt Conf Artif Intell 3:519–524

    Google Scholar 

  45. Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Hengel Avd (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1705–1714

  46. Mohamed S, Ejbali R, Zaied M (2019) Denoising autoencoder with dropout based network anomaly detection. ICSEA 2019:110

    Google Scholar 

  47. Bi M, Xu J, Wang M, Zhou F (2016) Anomaly detection model of user behavior based on principal component analysis. J Ambient Intell Humaniz Comput 7(4):547–554

    Article  Google Scholar 

  48. Liu FT, Ting KM, Zhou Z-H (2012) Isolation-based anomaly detection. ACM Trans Knowl Discov Data (TKDD) 6(1):1–39

    Article  Google Scholar 

  49. Kolberg J, Grimmer M, Gomez-Barrero M, Busch C (2021) Anomaly detection with convolutional autoencoders for fingerprint presentation attack detection. IEEE Trans Biometrics Behav Identity Sci 3(2):190–202

    Article  Google Scholar 

  50. Coelho G, Matos LM, Pereira PJ, Ferreira A, Pilastri A, Cortez P (2022) Deep autoencoders for acoustic anomaly detection: experiments with working machine and in-vehicle audio. Neural Comput Appl 1–15

Download references

Funding

The funding was provided by National Natural Science Foundation of China (No. U1933202), National Science and Technology Major Project (No. J2019-I-0001-0001), National Natural Science Foundation of China (No. 52105545), and Shandong Provincial Natural Science Foundation (No. ZR2020QE156).

Author information

Authors and Affiliations

  1. School of Mechatronics Engineering, Harbin Institute of Technology, Harbin, 150006, China

    Dan Liu, Shisheng Zhong & Lin Lin

  2. School of Ocean Engineering, Harbin Institute of Technology, Weihai, 264209, China

    Minghang Zhao, Xuyun Fu & Xueyun Liu

  3. Weihai Key Laboratory of Intelligent Operation and Maintenance, Harbin Institute of Technology, Weihai, 264209, China

    Shisheng Zhong, Minghang Zhao, Xuyun Fu & Xueyun Liu

Authors
  1. Dan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shisheng Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Minghang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuyun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xueyun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shisheng Zhong or Minghang Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Gas turbine data set experimental details

1.1 Architecture and hyperparameters of the OC-SVM

Table 10 shows how the choice of the hyperparameter nu specifically affect the AUROC(%) and ranking of the OC-SVM with different kernel functions. It can be seen that the optimal ranking of the OC-SVM with linear kernel is achieved when the nu is 0.001 for the gas turbine data set. The rest, respectively, are 0.0001, 0.01, and 0.0001, which is shown in Table 11.

Table 10 AUROC(%) and ranking of the OC-SVM on the validation set
Full size table
Table 11 Architecture and hyperparameters of the OC-SVM
Full size table

1.2 Architecture and hyperparameters of the GMM

Table 12 shows how the choice of the hyperparameter mixture components(mc) specifically affect the AUROC(%) and ranking of the GMM with different covariance types. It can be seen that the optimal ranking of the GMM with full covariance type is achieved when the number of mixture components is 512 for the gas turbine data set. The rest, respectively, are 96, 4, and 4, which is shown in Table 13.

Table 12 AUROC(%) and ranking of the GMM on the validation set
Full size table
Table 13 Architecture and hyperparameters of the GMM
Full size table

1.3 Architecture and hyperparameters of the DAE

Table 14 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the DAE. It can be seen that the optimal ranking of the DAE is achieved when the dimension of the latent representation is 55 for the gas turbine data set, which is shown in Table 15.

Table 14 AUROC(%) and ranking of the DAE on the validation set
Full size table
Table 15 Architecture and hyperparameters of the DAE
Full size table

1.4 Architecture and hyperparameters of the CAE

Table 16 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CAE. It can be seen that the optimal ranking of the CAE is achieved when the dimension of the latent representation is 128 for the gas turbine data set, which is shown in Table 17.

Table 16 AUROC(%) and ranking of the CAE on the validation set
Full size table
Table 17 Architecture and hyperparameters of the CAE
Full size table

1.5 Architecture and hyperparameters of the CSiamese

Table 18 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CSiamese. It can be seen that the optimal ranking of the CSiamese is achieved when the dimension of the latent representation is 128 for the gas turbine data set, which is shown in Table 19.

Table 18 AUROC(%) and ranking of the CSiamese on the validation set
Full size table
Table 19 Architecture and hyperparameters of the CSiamese
Full size table

Appendix B: CIFAR-10 experimental details

1.1 Architecture and hyperparameters of the DAE

Tables 20 and 21 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the DAE. It can be seen that the optimal ranking of the DAE is achieved when the dimension of the latent representation is 12 for the CIFAR-10 data set, which is shown in Table 22.

Table 20 AUROC(%)/Ranking of the DAE on the validation set
Full size table
Table 21 AUROC(%)/Ranking of the DAE on the validation set
Full size table
Table 22 Architecture and hyperparameters of the DAE
Full size table

1.2 Architecture and hyperparameters of the CAE

Tables 23 and 24 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CAE model. It can be seen that the optimal ranking of the CAE is achieved when the dimension of the latent representation is 512 for the CIFAR-10 data set, which is shown in Table 25.

Table 23 AUROC(%)/Ranking of the CAE on the validation setExample of a lengthy table which is set to full textwidth
Full size table
Table 24 AUROC(%)/Ranking of the CAE on the validation set
Full size table
Table 25 Architecture and hyperparameters of the CAE
Full size table

1.3 Architecture and hyperparameters of the CSiamese

Tables 26 and 27 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CSiamese. It can be seen that the optimal ranking of the CSiamese is achieved when the dimension of the latent representation is 6 for the CIFAR-10 data set, which is shown in Table 28.

Table 26 AUROC(%)/Ranking of the CSiamese on the validation set
Full size table
Table 27 AUROC(%)/Ranking of the CSiamese on the validation set
Full size table
Table 28 Architecture and hyperparameters of the CSiamese
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Zhong, S., Lin, L. et al. CSiamese: a novel semi-supervised anomaly detection framework for gas turbines via reconstruction similarity. Neural Comput & Applic 35, 16403–16427 (2023). https://doi.org/10.1007/s00521-023-08507-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08507-y

Keywords

Search

Navigation