Skip to main content

Advertisement

Dirichlet and Liouville-based normality scores for deep anomaly detection using transformations: applications to images and beyond images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

We address the problem of anomaly detection in data by learning a normality score function through the use of data transformations. Applying transformations to a dataset is essential for enhancing its representation and revealing underlying patterns. First, we propose geometric transformations for image data. The core idea of our approach is to train a multi-class deep classifier to distinguish between various geometric transformations. At test time, we construct the normality score function by approximating the softmax output predictions vector using generalized forms of Dirichlet distributions, including the generalized Dirichlet (GD), scaled Dirichlet (SD), shifted scaled Dirichlet (SSD), and Beta-Liouville (BL) distributions. These generalized forms of the Dirichlet distribution are more robust in real-world applications compared to the standard Dirichlet distribution. They offer a more flexible covariance structure, making them suitable for approximating both symmetric and asymmetric distributions. For parameter estimation, we use the maximum likelihood method based on the transformed forms of the original data. In the second step, we extend our approach to non-image data by selecting appropriate transformations. This transformation procedure involves building several neural networks, training them on the original data to obtain its transformed form, and then passing the transformed data through an auto-encoder. Experiments conducted on both image and non-image data demonstrate the effectiveness of our proposed strategy. The results show that our anomaly detection models, based on generalized Dirichlet distributions, outperform baseline techniques and achieve high Area Under the Receiver Operating Characteristic (AUROC) scores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of supporting data

Data could be made available on reasonable request.

References

  1. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: A survey. ACM Comput Surv (CSUR) 41(3):1–58

    Article  MATH  Google Scholar 

  2. Aleskerov E, Freisleben B, Rao B (1997) Cardwatch: A neural network based database mining system for credit card fraud detection. In: Proceedings of the IEEE/IAFE 1997 computational intelligence for financial engineering (CIFEr), pp 220–226. IEEE

  3. Spence C, Parra L, Sajda P (2001) Detection, synthesis and compression in mammographic image analysis with a hierarchical image probability model. In: Proceedings IEEE workshop on mathematical methods in biomedical image analysis (MMBIA 2001) (pp 3–10). IEEE

  4. Rousseeuw PJ, Leroy AM (2005) Robust regression and outlier detection. John wiley & sons

  5. Hodge V, Austin J (2004) A survey of outlier detection methodologies. Artif Intell Rev 22(2):85–126

    Article  MATH  Google Scholar 

  6. Mesgaran M, Hamza AB (2024) Graph fairing convolutional networks for anomaly detection. Pattern Recogn 145:109960

    Article  MATH  Google Scholar 

  7. Zhou K, Li J, Xiao Y, Yang J, Cheng J, Liu W ... Gao S (2021) Memorizing structure-texture correspondence for image anomaly detection. IEEE Trans Neural Netw Learn Syst

  8. Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Hengel AVD (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1705–1714

  9. Golan I, El-Yaniv R (2018) Deep anomaly detection using geometric transformations. Adv Neural Inf Process Syst 31

  10. Qiu C, Pfrommer T, Kloft M, Mandt S, Rudolph M (2021) Neural transformation learning for deep anomaly detection beyond images. In: International Conference on Machine Learning pp 8703–8714. PMLR

  11. Wong TT (1998) Generalized Dirichlet distribution in Bayesian analysis. Appl Math Comput 97(2–3):165–181

    MathSciNet  MATH  Google Scholar 

  12. Monti GS, Mateu i Figueras G, Pawlowsky-Glahn V, Egozcue JJ (2011) The shifted-scaled Dirichlet distribution in the simplex

  13. Lochner RH (1975) A Generalized Dirichlet distribution in Bayesian Life Testing. J Royal Stat Soc Series B (Methodological) 37:103–113

    Article  MathSciNet  MATH  Google Scholar 

  14. Baur C, Wiestler B, Albarqouni S, Navab N (2018) Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: International MICCAI brainlesion workshop (pp 161–169). Springer, Cham

  15. Zhou C, Paffenroth RC (2017) Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 665–674

  16. Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging, pp 146–157. Springer, Cham

  17. Yeh RA, Chen C, Yian Lim T, Schwing AG, Hasegawa-Johnson M, Do MN (2017) Semantic image inpainting with deep generative models. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5485–5493

  18. Zhou K, Li J, Luo W, Li Z, Yang J, Fu H, ... Gao S (2021) Proxy-bridged Image Reconstruction Network for Anomaly Detection in Medical Images. IEEE Trans Med Imaging 41(3):582–594

  19. Pidhorskyi S, Almohsen R, Doretto G (2018) Generative probabilistic novelty detection with adversarial autoencoders. Adv Neural Inf Process Syst 31

  20. Jolliffe I (2002) Principal Component Analysis. Allemagne: Springer

  21. Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? Journal of the ACM (JACM) 58(3):1–37

    Article  MathSciNet  MATH  Google Scholar 

  22. Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning (Vol 4, No. 4, p 738). New York: springer

  23. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, pp 1597–1607. PMLR

  24. Ouardini K, Yang H, Unnikrishnan B, Romain M, Garcin C, Zenati H, ... Foo CS (2019) Towards practical unsupervised anomaly detection on retinal images. In: Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data, pp 225–234. Springer, Cham

  25. Hautamaki V, Karkkainen I, Franti P (2004) Outlier detection using k-nearest neighbour graph. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol 3, pp 430–433. IEEE

  26. Malhotra P, Ramakrishnan A, Anand G, Vig L, Agarwal P, Shroff G (2016) LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.00148

  27. Kim TY, Cho SB (2018) Web traffic anomaly detection using C-LSTM neural networks. Expert Syst Appl 106:66–76

    Article  MATH  Google Scholar 

  28. Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv preprint arXiv:1605.07146

  29. Minka T (2000) Estimating a Dirichlet distribution

  30. Bouguila N, Ziou D (2007) High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans Pattern Anal Mach Intell 29(10):1716–1731

    Article  MATH  Google Scholar 

  31. Oboh BS, Bouguila N (2017) Unsupervised learning of finite mixtures using scaled dirichlet distribution and its application to software modules categorization. In: 2017 IEEE international conference on industrial technology (ICIT), pp 1085–1090. IEEE

  32. Zamzami N, Bouguila N (2020) Probabilistic modeling for frequency vectors using a flexible shifted-scaled Dirichlet distribution prior. ACM Transactions on Knowledge Discovery from Data (TKDD) 14(6):1–35

    Article  MATH  Google Scholar 

  33. Bourouis S, Alharbi A, Bouguila N (2021) Bayesian learning of shifted-scaled dirichlet mixture models and its application to early COVID-19 detection in chest X-ray images. J Imag 7(1):7

    Article  MATH  Google Scholar 

  34. Bouguila N (2011) Hybrid generative/discriminative approaches for proportional data modeling and classification. IEEE Trans Knowl Data Eng 24(12):2184–2202

    Article  MATH  Google Scholar 

  35. Bouguila N (2012) Infinite Liouville mixture models with application to text and texture categorization. Pattern Recogn Lett 33(2):103–110

    Article  MATH  Google Scholar 

  36. Krizhevsky A, Hinton G (2010) Convolutional deep belief networks on cifar-10. Unpublished manuscript 40(7):1–9

  37. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  38. Wicker N, Muller J, Kalathur RKR, Poch O (2008) A maximum likelihood approximation method for Dirichlet’s parameter estimation. Comput Stat Data Anal 52(3):1315–1322

    Article  MathSciNet  MATH  Google Scholar 

  39. Deecke L, Vandermeulen R, Ruff L, Mandt S, Kloft M (2019) Image anomaly detection with generative adversarial networks. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, Part I 18, pp 3–17. Springer International Publishing

  40. Zhai S, Cheng Y, Lu W, Zhang Z (2016) Deep structured energy based models for anomaly detection. In International conference on machine learning, pp 1100–1109. PMLR

  41. Ferrari C, Foca G, Ulrici A (2013) Handling large datasets of hyperspectral images: Reducing data size without loss of useful information. Anal Chim Acta 802:29–39

    Article  Google Scholar 

  42. Sghaier O (2023) Enhancing Anomaly Detection with Flexible Distribution Models (Doctoral dissertation, Concordia University)

  43. Sghaier O, Amayri M, Bouguila N (2024) Libby-Novick Beta-Liouville Distribution for Enhanced Anomaly Detection in Proportional Data. ACM Trans Intell Syst Technol

  44. Sghaier O, Amayri M, Bouguila N (2023) Multivariate Beta Normality Scores Approach for Deep Anomaly Detection in Images Using Transformations. In: 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp 3428–3433. IEEE

Download references

Funding

This work was funded via NSERC project number 6656-2017.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nizar Bouguila.

Ethics declarations

Ethical Approval

Not applicable

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

When working with highly imbalanced datasets, precision-recall curves (PRC) can often provide more meaningful insights than the ROC curve. To ensure thoroughness, we present the performance of all baseline models in terms of the area under the precision-recall curve (AUPR). The following table summarizes the AUPR results for the three image datasets.

Table 10 Performance Comparison (Accuracy %)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sghaier, O., Amayri, M. & Bouguila, N. Dirichlet and Liouville-based normality scores for deep anomaly detection using transformations: applications to images and beyond images. Appl Intell 55, 25 (2025). https://doi.org/10.1007/s10489-024-05892-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05892-2

Keywords