Skip to main content
SpringerLink
Log in

VESC: a new variational autoencoder based model for anomaly detection

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Anomaly detection is a hot and practical problem. Most of the existing research is based on the model of the generative model, which judges abnormalities by comparing the data errors between original samples and reconstruction samples. Among them, Variational AutoEncoder (VAE) is widely used, but it has the problem of over-generalization. In this paper, we design an unsupervised deep learning anomaly detection method named VESC and propose the recursive reconstruction strategy. VESC adopts the idea of data compression and three structures on the basis of the original VAE, namely spatial constrained network, reformer structure, and re-encoder. The recursive reconstruction strategy can improve the accuracy of the model by increasing the number and typicality of training samples, and it can apply to most unsupervised learning methods. Experimental results of several benchmarks show that our model outperforms state-of-the-art anomaly detection methods. And our proposed strategy can improve the detection results of the original model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Onat I, Miri A (2017) An intrusion detection system for wireless sensor networks. In: International conference on telecommunications

  2. Pozzolo AD, Boracchi G, Caelen O, Alippi C, Bontempi G (2018) Credit card fraud detection: a realistic modeling and a novel learning strategy. IEEE Trans Neural Netw Learn Syst 29(8):3784–3797

    Article  Google Scholar 

  3. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery

  4. Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: a survey. arXiv:1901.03407

  5. Bayer J, Osendorfer C (2015) Learning stochastic recurrent networks

  6. Pham N, Pagh R (2012) A near-linear time approximation algorithm for angle-based outlier detection in high-dimensional data. ACM, pp 877–885

  7. Pham N (2018) L1-depth revisited: a robust angle-based outlier factor in high-dimensional space. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 105–121

  8. Sathe S, Aggarwal CC (2018) Subspace histograms for outlier detection in linear time. Knowl Inf Syst 1–25

  9. Kim C, Lee J, Kim R, Park Y, Kang J (2018) Deepnap: deep neural anomaly pre-detection in a semiconductor fab. Inf Sci 457:002002551830375

    MathSciNet  Google Scholar 

  10. Pukelsheim F (1994) The three sigma rule. Am Stat 48(2):88–91

    MathSciNet  Google Scholar 

  11. Grubbs FE (1950) Sample criteria for testing outlying observations. Ann Math Stat 21(1):27–58. https://doi.org/10.1214/aoms/1177729885

    Article  MathSciNet  MATH  Google Scholar 

  12. Munawar A, Vinayavekhin P, De Magistris G (2017) Limiting the reconstruction capability of generative neural network using negative learning. In: 2017 IEEE 27th international workshop on machine learning for signal processing (MLSP), pp 1–6. https://doi.org/10.1109/MLSP.2017.8168155

  13. Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Van Den Hengel A (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 1705–1714. https://doi.org/10.1109/ICCV.2019.00179

  14. Kieu T, Yang B, Jensen CS (2018) Outlier detection for multidimensional time series using deep neural networks. In: 2018 19th IEEE international conference on mobile data management (MDM), pp 125–134. https://doi.org/10.1109/MDM.2018.00029

  15. Zong B, Song Q, Min MR, Cheng W, Lumezanu C, Cho D, Chen H (2018) Deep autoencoding gaussian mixture model for unsupervised anomaly detection

  16. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  17. Schlegl T, Seeböck P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) f-anogan: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44

    Article  Google Scholar 

  18. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. Curran Associates, Inc.

  19. Hicsonmez S, Samet N, Akbas E, Duygulu P (2020) GANILLA: generative adversarial networks for image to illustration translation. CoRR. arXiv:2002.05638

  20. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv:1312.6114

  21. Liu G, Guo C, Xie L, Liu W, Xiong NN, Chen G (2020) An intelligent cnn-vae text representation technology based on text semantics for comprehensive big data. arXiv:abs/2008.12522

  22. Cui Z, Wang J, Bai B, Guo T, Feng Y (2020) G-vae: a continuously variable rate deep image compression framework. arXiv:abs/2003.02012

  23. Lin S, Clark R, Birke R, Schönborn S, Trigoni N, Roberts SJ (2020) Anomaly detection for time series using vae-lstm hybrid model. ICASSP 2020—2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4322–4326

  24. An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability

  25. Li Z, Chen W, Pei D (2018) Robust and unsupervised kpi anomaly detection based on conditional variational autoencoder, pp 1–9. https://doi.org/10.1109/PCCC.2018.8710885

  26. Tu Z, Bai X (2009) Auto-context and its application to high-level vision tasks and 3d brain image segmentation. IEEE Trans Pattern Anal Mach Intell 32(10):1744–1757

    Google Scholar 

  27. Wang W, Yu K, Hugonot J, Fua P, Salzmann M (2019) Recurrent u-net for resource-constrained segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 2142–2151

  28. Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154–6162

  29. Dau HA, Keogh E, Kamgar K, Yeh C-CM, Zhu Y, Gharghabi S, Ratanamahatana CA, Yanping, Hu B, Begum N, Bagnall A, Mueen A, Batista G (2018) The UCR time series classification archive. https://www.cs.ucr.edu/~eamonn/time_series_data_2018/

  30. Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  31. Liu FT, Kai MT, Zhou ZH (2009) Isolation forest. In: Eighth IEEE international conference on data mining

  32. Erfani SM, Rajasegarar S, Karunasekera S, Leckie C (2016) High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recognit 58:121–134

    Article  Google Scholar 

  33. Zenati H, Romain M, Foo CS, Lecouat B, Chandrasekhar VR (2018) Adversarially learned anomaly detection

  34. An J, Cho S (2015) Variational autoencoder based anomaly detection using reconstruction probability. Special lecture on IE 2(1)

  35. Donahue J, Krähenbühl P, Darrell T (2016) Adversarial feature learning. arXiv:1605.09782

  36. Akcay S, Atapour-Abarghouei A, Breckon TP (2018) Ganomaly: semi-supervised anomaly detection via adversarial training. In: Asian conference on computer vision, pp 622–637

  37. Ling CX, Huang J, Zhang H (2003) Auc: a statistically consistent and more discriminating measure than accuracy. In: International joint conference on artificial intelligence

  38. Akçay S, Atapour-Abarghouei A, Breckon TP (2019) Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In: 2019 international joint conference on neural networks (IJCNN). IEEE, pp 1–8

Download references

Acknowledgements

This work was supported by Natural Science Foundation of Guangdong Province, China (Grant No. 2020A1515010970) and Shenzhen Research Council (Grant Nos. JCYJ20200109113427092, GJHZ20180928155209705).

Author information

Authors and Affiliations

  1. Harbin Institute of Technology Shenzhen, Shenzhen, 518000, China

    Chunkai Zhang, Xinyu Wang, Jiahua Zhang, Shaocong Li, Chuanyi Liu & Peiyi Han

  2. New York Stock Exchange LLC, Wall Street, New York, 10005, USA

    Hanyu Zhang

Authors
  1. Chunkai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiahua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaocong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hanyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chuanyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Peiyi Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chunkai Zhang or Peiyi Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Wang, X., Zhang, J. et al. VESC: a new variational autoencoder based model for anomaly detection. Int. J. Mach. Learn. & Cyber. 14, 683–696 (2023). https://doi.org/10.1007/s13042-022-01657-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01657-w

Keywords

Search

Navigation