Abstract
The main problem in data-driven anomaly detection of gas turbines is that the monitoring data consists of only a very small number of abnormal samples with the overwhelming majority of normal samples. To address this problem, this paper develops a novel semi-supervised anomaly detection framework, namely CSiamese, and the parameters of the framework are optimized only using normal samples, with the ultimate purpose of improving the anomaly detection performance on imbalanced data sets. First, the convolutional auto-encoder is used to learn the reconstructed representation of the input sample. Second, the Siamese network is selected to learn to measure the similarity between the input and its reconstructed representation under noise conditions. Besides, a new loss function is developed by improving the contrastive loss, namely triangle loss, and can reduce the risk of collapsing solutions of the Siamese network when only using positive sample pairs. Third, maximum likelihood estimation is used to set the proper detection threshold to separate abnormal samples from normal samples. Finally, the effectiveness of the developed CSiamese has been evaluated using the real monitoring data of gas turbines and a public CIFAR-10 data set.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Tahan M, Tsoutsanis E, Muhammad M, Karim ZA (2017) Performance-based health monitoring, diagnostics and prognostics for condition-based maintenance of gas turbines: a review. Appl Energy 198:122–144
Yan W, Yu L (2019) On accurate and reliable anomaly detection for gas turbine combustors: a deep learning approach. arXiv preprint arXiv:1908.09238
Yazdani S, Montazeri-Gh M (2020) A novel gas turbine fault detection and identification strategy based on hybrid dimensionality reduction and uncertain rule-based fuzzy logic. Comput Ind 115:103–131
Mousavi M, Chaibakhsh A, Jamali A, Kordestani M, Saif M (2022) A new fault diagnosis approach for heavy-duty gas turbines. IEEE/ASME Trans Mechatron
Asgari H, Chen X, Sainudiin R (2013) Modelling and simulation of gas turbines. Int J Model Ident Control 20(3):253–270
Simon DL, Rinehart AW (2014) A model-based anomaly detection approach for analyzing streaming aircraft engine measurement data, vol 45752. American Society of Mechanical Engineers
Tian Z, Jin T, Wu B, Ding F (2011) Condition based maintenance optimization for wind power generation systems under continuous monitoring. Renew Energy 36(5):1502–1509
Li X, Ding Q, Sun J (2018) Remaining useful life estimation in prognostics using deep convolution neural networks. Reliabil Eng Syst Saf 172:1–11
Tang Z, Chen Z, Bao Y, Li H (2019) Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Health Monit 26(1):2296
Garg S, Kaur K, Kumar N, Kaddoum G, Zomaya AY, Ranjan R (2019) A hybrid deep learning-based model for anomaly detection in cloud datacenter networks. IEEE Trans Netw Serv Manag 16(3):924–935
Zhong S, Liu D, Lin L, Zhao M, Fu X, Guo F (2020) A novel anomaly detection method for gas turbines using weight agnostic neural network search. Asia-Pacific international symposium on advanced reliability and maintenance modeling, pp 1–6. IEEE
Zhou X, Liang W, Shimizu S, Ma J, Jin Q (2020) Siamese neural network based few-shot learning for anomaly detection in industrial cyber-physical systems. IEEE Trans Ind Inf 17(8):5790–5798
Meng L, Zhao M, Cui Z, Zhang X, Zhong S (2022) Empirical mode reconstruction: Preserving intrinsic components in data augmentation for intelligent fault diagnosis of civil aviation hydraulic pumps. Comput Ind 134:103557
Yoo Y (2020) Data-driven fault detection process using correlation based clustering. Comput Ind 122:103–279
Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: a survey. arXiv preprint arXiv:1901.03407
Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016) High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recogn 58:121–134
Zhu L, Borlaug B, Lin L, Holden J, Gonder J (2022) Light-duty vehicle trip classification using one-class novelty detection and exhaustive feature extraction. IEEE Trans Ind Appl
Duarte DP, Nogueira RN, Bilro LB (2019) Semi-supervised gaussian and t-distribution hybrid mixture model for water leak detection. Meas Sci Technol 30(12):125109
Hyunseong L, Hyung JL, Aditi C (2021) Data-driven system health monitoring technique using autoencoder for the safety management of commercial aircraft. Neural Comput Appl 33:3235–3250
Yan W (2020) Detecting gas turbine combustor anomalies using semi-supervised anomaly detection with deep representation learning. Cogn Comput 12(2):398–411
Ribeiro M, Lazzaretti AE, Lopes HS (2018) A study of deep convolutional auto-encoders for anomaly detection in videos. Pattern Recogn Lett 105:13–22
Xuyun F, Hui L, Zhong S, Lin L (2019) Aircraft engine fault detection based on grouped convolutional denoising autoencoders. Chin J Aeronaut 32(2):296–307
Turchenko V, Luczak A (2017) Creation of a deep convolutional auto-encoder in caffe. In: IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, vol 2. IEEE, pp 651–659
Suganuma M, Ozay M, Okatani T (2018) Exploiting the potential of standard convolutional autoencoders for image restoration by evolutionary search. International Conference on Machine Learning. PMLR, pp 4771–4780
Ren Y, Wang N, Li M, Xu Z (2020) Deep density-based image clustering. Knowl-Based Syst 197:105841
Hamdi S, Snoussi H, Abid M (2020) Fine-tuning a pre-trained cae for deep one class anomaly detection in video footage. In: Mediterranean conference on pattern recognition and artificial intelligence. Springer, pp 3–17
Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, pp 1735–1742
Wang Z, Bovik AC (2009) Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98–117
Larsen ABL, Sonderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. International Conference on Machine Learning. PMLR, pp 1558–1566
Zhou C, Paffenroth RC (2017) Anomaly detection with robust deep autoencoders. Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 665–674
Sohangir S, Wang D (2017) Improved sqrt-cosine similarity measurement. J Big Data 4(1):1–13
Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. IEEE computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp 539–546
Liu D, Zhong S, Lin L, Zhao M, Xia X, Fu X, Cui Z (2021) A novel performance prediction method for gas turbines using the prophet model. In: 2021 IEEE international conference on sensing, diagnostics, prognostics, and control (SDPC). IEEE, pp 203–208
Li Z, Zhong S, Lin L (2017) Novel gas turbine fault diagnosis method based on performance deviation model. J Propul Power 33(3):730–739
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434
Chow JK, Su Z, Wu J, Tan PS, Mao X, Wang Y-H (2020) Anomaly detection of defects on concrete structures with the convolutional autoencoder. Adv Eng Inform 45:101105
Finke T, Kramer M, Morandini A, Muck A, Oleksiyuk I (2021) Autoencoders for unsupervised anomaly detection in high energy physics. J High Energy Phys 2021(6):1–32
Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134
Bromley J, Guyon I, LeCun Y, Sackinger E, Shah R (1993) Signature verification using a siamese time delay neural network. Adv Neural Inf Process Syst 6:737–744
Ahrabian K, BabaAli B (2019) Usage of autoencoders and siamese networks for online handwritten signature verification. Neural Comput Appl 31:9321–9334
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66
Compare M, Martini F, Mattafirri S, Carlevaro F, Zio E (2016) Semi-Markov model for the oxidation degradation mechanism in gas turbine nozzles. IEEE Trans Reliab 65(2):574–581
Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. Asian Conference on Computer Vision. Springer, pp 622–637
Ling CX, Huang J, Zhang H et al (2003) Auc: a statistically consistent and more discriminating measure than accuracy. Int Jt Conf Artif Intell 3:519–524
Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Hengel Avd (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1705–1714
Mohamed S, Ejbali R, Zaied M (2019) Denoising autoencoder with dropout based network anomaly detection. ICSEA 2019:110
Bi M, Xu J, Wang M, Zhou F (2016) Anomaly detection model of user behavior based on principal component analysis. J Ambient Intell Humaniz Comput 7(4):547–554
Liu FT, Ting KM, Zhou Z-H (2012) Isolation-based anomaly detection. ACM Trans Knowl Discov Data (TKDD) 6(1):1–39
Kolberg J, Grimmer M, Gomez-Barrero M, Busch C (2021) Anomaly detection with convolutional autoencoders for fingerprint presentation attack detection. IEEE Trans Biometrics Behav Identity Sci 3(2):190–202
Coelho G, Matos LM, Pereira PJ, Ferreira A, Pilastri A, Cortez P (2022) Deep autoencoders for acoustic anomaly detection: experiments with working machine and in-vehicle audio. Neural Comput Appl 1–15
Funding
The funding was provided by National Natural Science Foundation of China (No. U1933202), National Science and Technology Major Project (No. J2019-I-0001-0001), National Natural Science Foundation of China (No. 52105545), and Shandong Provincial Natural Science Foundation (No. ZR2020QE156).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Gas turbine data set experimental details
1.1 Architecture and hyperparameters of the OC-SVM
Table 10 shows how the choice of the hyperparameter nu specifically affect the AUROC(%) and ranking of the OC-SVM with different kernel functions. It can be seen that the optimal ranking of the OC-SVM with linear kernel is achieved when the nu is 0.001 for the gas turbine data set. The rest, respectively, are 0.0001, 0.01, and 0.0001, which is shown in Table 11.
1.2 Architecture and hyperparameters of the GMM
Table 12 shows how the choice of the hyperparameter mixture components(mc) specifically affect the AUROC(%) and ranking of the GMM with different covariance types. It can be seen that the optimal ranking of the GMM with full covariance type is achieved when the number of mixture components is 512 for the gas turbine data set. The rest, respectively, are 96, 4, and 4, which is shown in Table 13.
1.3 Architecture and hyperparameters of the DAE
Table 14 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the DAE. It can be seen that the optimal ranking of the DAE is achieved when the dimension of the latent representation is 55 for the gas turbine data set, which is shown in Table 15.
1.4 Architecture and hyperparameters of the CAE
Table 16 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CAE. It can be seen that the optimal ranking of the CAE is achieved when the dimension of the latent representation is 128 for the gas turbine data set, which is shown in Table 17.
1.5 Architecture and hyperparameters of the CSiamese
Table 18 shows how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CSiamese. It can be seen that the optimal ranking of the CSiamese is achieved when the dimension of the latent representation is 128 for the gas turbine data set, which is shown in Table 19.
Appendix B: CIFAR-10 experimental details
1.1 Architecture and hyperparameters of the DAE
Tables 20 and 21 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the DAE. It can be seen that the optimal ranking of the DAE is achieved when the dimension of the latent representation is 12 for the CIFAR-10 data set, which is shown in Table 22.
1.2 Architecture and hyperparameters of the CAE
Tables 23 and 24 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CAE model. It can be seen that the optimal ranking of the CAE is achieved when the dimension of the latent representation is 512 for the CIFAR-10 data set, which is shown in Table 25.
1.3 Architecture and hyperparameters of the CSiamese
Tables 26 and 27 depict how the choice of the hyperparameter specifically affect the AUROC(%) and ranking of the CSiamese. It can be seen that the optimal ranking of the CSiamese is achieved when the dimension of the latent representation is 6 for the CIFAR-10 data set, which is shown in Table 28.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, D., Zhong, S., Lin, L. et al. CSiamese: a novel semi-supervised anomaly detection framework for gas turbines via reconstruction similarity. Neural Comput & Applic 35, 16403–16427 (2023). https://doi.org/10.1007/s00521-023-08507-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-023-08507-y