Skip to main content
SpringerLink
Log in

Latent Diffusion Based Multi-class Anomaly Detection

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14361))

Included in the following conference series:

  • 504 Accesses

Abstract

The unsupervised anomaly detection problem holds great importance but remains challenging to address due to the myriad of data possibilities in our daily lives. Currently, distinct models are trained for different scenarios. In this work, we introduce a reconstruction-based anomaly detection structure built on the Latent Space Denoising Diffusion Probabilistic Model (LDM). This structure effectively detects anomalies in multi-class situations. When normal data comprises multiple object categories, existing reconstruction models often learn identical patterns. This leads to the successful reconstruction of both normal and anomalous data based on these patterns, resulting in the inability to distinguish anomalous data. To address this limitation, we implemented the LDM model. Its process of adding noise effectively disrupts identical patterns. Additionally, this advanced image generation model can generate images that deviate from the input. We have further proposed a classification model that compares the input with the reconstruction results, tapping into the generative power of the LDM model. Our structure has been tested on the MNIST and CIFAR-10 datasets, where it surpassed the performance of state-of-the-art reconstruction-based anomaly detection models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223. PMLR (2017)

    Google Scholar 

  2. Boukerche, A., Zheng, L., Alfandi, O.: Outlier detection: methods, models, and classification. ACM Comput. Surv. (CSUR) 53(3), 1–37 (2020)

    Article  Google Scholar 

  3. Chalapathy, R., Menon, A.K., Chawla, S.: Robust, deep and inductive anomaly detection. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 36–51. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_3

    Chapter  Google Scholar 

  4. Chen, J., Sathe, S., Aggarwal, C., Turaga, D.: Outlier detection with autoencoder ensembles. In: Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 90–98. SIAM (2017)

    Google Scholar 

  5. Dasgupta, D., Nino, F.: A comparison of negative and positive selection algorithms in novel pattern detection. In: SMC 2000 Conference Proceedings. 2000 IEEE International Conference on Systems, Man and Cybernetics. Cybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions cat. no. 0, vol. 1, pp. 125–130. IEEE (2000)

    Google Scholar 

  6. Deecke, L., Ruff, L., Vandermeulen, R.A., Bilen, H.: Transfer-based semantic anomaly detection. In: International Conference on Machine Learning, pp. 2546–2558. PMLR (2021)

    Google Scholar 

  7. Eskin, E.: Anomaly detection over noisy data using learned probability distributions (2000)

    Google Scholar 

  8. Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873–12883 (2021)

    Google Scholar 

  9. Graham, M.S., Pinaya, W.H., Tudosiu, P.D., Nachev, P., Ourselin, S., Cardoso, J.: Denoising diffusion models for out-of-distribution detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2947–2956 (2023)

    Google Scholar 

  10. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  11. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  13. Laurikkala, J., Juhola, M., Kentala, E., Lavrac, N., Miksch, S., Kavsek, B.: Informal identification of outliers in medical data. In: Fifth International Workshop on Intelligent Data Analysis in Medicine and Pharmacology, vol. 1, pp. 20–24. Citeseer (2000)

    Google Scholar 

  14. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  15. Lu, W., et al.: Unsupervised sequential outlier detection with deep architectures. IEEE Trans. Image Process. 26(9), 4321–4330 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  16. Makhzani, A., Frey, B.: K-sparse autoencoders. arXiv preprint arXiv:1312.5663 (2013)

  17. Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. (CSUR) 54(2), 1–38 (2021)

    Article  Google Scholar 

  18. Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contractive auto-encoders: explicit invariance during feature extraction. In: ICML (2011)

    Google Scholar 

  19. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695 (2022)

    Google Scholar 

  20. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., Langs, G.: Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 146–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_12

    Chapter  Google Scholar 

  21. Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)

    Article  MATH  Google Scholar 

  22. Shewhart, W.A.: Economic Control of Quality of Manufactured Product. Macmillan And Co Ltd, London (1931)

    Google Scholar 

  23. Tax, D.M., Duin, R.P.: Support vector data description. Mach. Learn. 54(1), 45–66 (2004)

    Article  MATH  Google Scholar 

  24. Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. arXiv preprint arXiv:1703.00395 (2017)

  25. Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  26. Velleman, P.F., Hoaglin, D.C.: Applications, Basics, and Computing of Exploratory Data Analysis. Duxbury Press, New York (1981)

    Google Scholar 

  27. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A., Bottou, L.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(12) (2010)

    Google Scholar 

  28. Wang, S., Wu, L., Cui, L., Shen, Y.: Glancing at the patch: anomaly localization with global and local feature comparison. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 254–263 (2021)

    Google Scholar 

  29. Xia, Y., Zhang, Y., Liu, F., Shen, W., Yuille, A.L.: Synthesize then compare: detecting failures and anomalies for semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 145–161. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_9

    Chapter  Google Scholar 

  30. You, Z., et al.: A unified model for multi-class anomaly detection. Adv. Neural. Inf. Process. Syst. 35, 4571–4584 (2022)

    Google Scholar 

  31. Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient GAN-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)

  32. Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 665–674 (2017)

    Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported in part by the National Science Foundation under grant numbers [OAC-2201599] and the National Institute of General Medical Sciences of the National Institutes of Health under grant numbers [P30 GM145646].

Author information

Authors and Affiliations

  1. University of Nevada, Reno, Reno, NV, 89512, USA

    Chenxing Wang & Alireza Tavakkoli

Authors
  1. Chenxing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Tavakkoli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenxing Wang .

Editor information

Editors and Affiliations

  1. University of Nevada Reno, Reno, NV, USA

    George Bebis

  2. Google Research, Mountain View, CA, USA

    Golnaz Ghiasi

  3. New York University, New York, USA

    Yi Fang

  4. Ben-Gurion University, Be'er Sheva, Israel

    Andrei Sharf

  5. Microsoft Research, Beijing, China

    Yue Dong

  6. The University of Oklahoma, Norman, OK, USA

    Chris Weaver

  7. University of Maryland, Collage Park, MD, USA

    Zhicheng Leo

  8. University of Central Florida, Orlando, FL, USA

    Joseph J. LaViola Jr.

  9. InnerOptic Technology, Hillsborough, NC, USA

    Luv Kohli

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, C., Tavakkoli, A. (2023). Latent Diffusion Based Multi-class Anomaly Detection. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14361. Springer, Cham. https://doi.org/10.1007/978-3-031-47969-4_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47969-4_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47968-7

  • Online ISBN: 978-3-031-47969-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation