Abstract
We address the problem of anomaly detection in data by learning a normality score function through the use of data transformations. Applying transformations to a dataset is essential for enhancing its representation and revealing underlying patterns. First, we propose geometric transformations for image data. The core idea of our approach is to train a multi-class deep classifier to distinguish between various geometric transformations. At test time, we construct the normality score function by approximating the softmax output predictions vector using generalized forms of Dirichlet distributions, including the generalized Dirichlet (GD), scaled Dirichlet (SD), shifted scaled Dirichlet (SSD), and Beta-Liouville (BL) distributions. These generalized forms of the Dirichlet distribution are more robust in real-world applications compared to the standard Dirichlet distribution. They offer a more flexible covariance structure, making them suitable for approximating both symmetric and asymmetric distributions. For parameter estimation, we use the maximum likelihood method based on the transformed forms of the original data. In the second step, we extend our approach to non-image data by selecting appropriate transformations. This transformation procedure involves building several neural networks, training them on the original data to obtain its transformed form, and then passing the transformed data through an auto-encoder. Experiments conducted on both image and non-image data demonstrate the effectiveness of our proposed strategy. The results show that our anomaly detection models, based on generalized Dirichlet distributions, outperform baseline techniques and achieve high Area Under the Receiver Operating Characteristic (AUROC) scores.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of supporting data
Data could be made available on reasonable request.
References
Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: A survey. ACM Comput Surv (CSUR) 41(3):1–58
Aleskerov E, Freisleben B, Rao B (1997) Cardwatch: A neural network based database mining system for credit card fraud detection. In: Proceedings of the IEEE/IAFE 1997 computational intelligence for financial engineering (CIFEr), pp 220–226. IEEE
Spence C, Parra L, Sajda P (2001) Detection, synthesis and compression in mammographic image analysis with a hierarchical image probability model. In: Proceedings IEEE workshop on mathematical methods in biomedical image analysis (MMBIA 2001) (pp 3–10). IEEE
Rousseeuw PJ, Leroy AM (2005) Robust regression and outlier detection. John wiley & sons
Hodge V, Austin J (2004) A survey of outlier detection methodologies. Artif Intell Rev 22(2):85–126
Mesgaran M, Hamza AB (2024) Graph fairing convolutional networks for anomaly detection. Pattern Recogn 145:109960
Zhou K, Li J, Xiao Y, Yang J, Cheng J, Liu W ... Gao S (2021) Memorizing structure-texture correspondence for image anomaly detection. IEEE Trans Neural Netw Learn Syst
Gong D, Liu L, Le V, Saha B, Mansour MR, Venkatesh S, Hengel AVD (2019) Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1705–1714
Golan I, El-Yaniv R (2018) Deep anomaly detection using geometric transformations. Adv Neural Inf Process Syst 31
Qiu C, Pfrommer T, Kloft M, Mandt S, Rudolph M (2021) Neural transformation learning for deep anomaly detection beyond images. In: International Conference on Machine Learning pp 8703–8714. PMLR
Wong TT (1998) Generalized Dirichlet distribution in Bayesian analysis. Appl Math Comput 97(2–3):165–181
Monti GS, Mateu i Figueras G, Pawlowsky-Glahn V, Egozcue JJ (2011) The shifted-scaled Dirichlet distribution in the simplex
Lochner RH (1975) A Generalized Dirichlet distribution in Bayesian Life Testing. J Royal Stat Soc Series B (Methodological) 37:103–113
Baur C, Wiestler B, Albarqouni S, Navab N (2018) Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: International MICCAI brainlesion workshop (pp 161–169). Springer, Cham
Zhou C, Paffenroth RC (2017) Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 665–674
Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging, pp 146–157. Springer, Cham
Yeh RA, Chen C, Yian Lim T, Schwing AG, Hasegawa-Johnson M, Do MN (2017) Semantic image inpainting with deep generative models. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5485–5493
Zhou K, Li J, Luo W, Li Z, Yang J, Fu H, ... Gao S (2021) Proxy-bridged Image Reconstruction Network for Anomaly Detection in Medical Images. IEEE Trans Med Imaging 41(3):582–594
Pidhorskyi S, Almohsen R, Doretto G (2018) Generative probabilistic novelty detection with adversarial autoencoders. Adv Neural Inf Process Syst 31
Jolliffe I (2002) Principal Component Analysis. Allemagne: Springer
Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? Journal of the ACM (JACM) 58(3):1–37
Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning (Vol 4, No. 4, p 738). New York: springer
Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, pp 1597–1607. PMLR
Ouardini K, Yang H, Unnikrishnan B, Romain M, Garcin C, Zenati H, ... Foo CS (2019) Towards practical unsupervised anomaly detection on retinal images. In: Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data, pp 225–234. Springer, Cham
Hautamaki V, Karkkainen I, Franti P (2004) Outlier detection using k-nearest neighbour graph. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol 3, pp 430–433. IEEE
Malhotra P, Ramakrishnan A, Anand G, Vig L, Agarwal P, Shroff G (2016) LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.00148
Kim TY, Cho SB (2018) Web traffic anomaly detection using C-LSTM neural networks. Expert Syst Appl 106:66–76
Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv preprint arXiv:1605.07146
Minka T (2000) Estimating a Dirichlet distribution
Bouguila N, Ziou D (2007) High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans Pattern Anal Mach Intell 29(10):1716–1731
Oboh BS, Bouguila N (2017) Unsupervised learning of finite mixtures using scaled dirichlet distribution and its application to software modules categorization. In: 2017 IEEE international conference on industrial technology (ICIT), pp 1085–1090. IEEE
Zamzami N, Bouguila N (2020) Probabilistic modeling for frequency vectors using a flexible shifted-scaled Dirichlet distribution prior. ACM Transactions on Knowledge Discovery from Data (TKDD) 14(6):1–35
Bourouis S, Alharbi A, Bouguila N (2021) Bayesian learning of shifted-scaled dirichlet mixture models and its application to early COVID-19 detection in chest X-ray images. J Imag 7(1):7
Bouguila N (2011) Hybrid generative/discriminative approaches for proportional data modeling and classification. IEEE Trans Knowl Data Eng 24(12):2184–2202
Bouguila N (2012) Infinite Liouville mixture models with application to text and texture categorization. Pattern Recogn Lett 33(2):103–110
Krizhevsky A, Hinton G (2010) Convolutional deep belief networks on cifar-10. Unpublished manuscript 40(7):1–9
Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747
Wicker N, Muller J, Kalathur RKR, Poch O (2008) A maximum likelihood approximation method for Dirichlet’s parameter estimation. Comput Stat Data Anal 52(3):1315–1322
Deecke L, Vandermeulen R, Ruff L, Mandt S, Kloft M (2019) Image anomaly detection with generative adversarial networks. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10-14, 2018, Proceedings, Part I 18, pp 3–17. Springer International Publishing
Zhai S, Cheng Y, Lu W, Zhang Z (2016) Deep structured energy based models for anomaly detection. In International conference on machine learning, pp 1100–1109. PMLR
Ferrari C, Foca G, Ulrici A (2013) Handling large datasets of hyperspectral images: Reducing data size without loss of useful information. Anal Chim Acta 802:29–39
Sghaier O (2023) Enhancing Anomaly Detection with Flexible Distribution Models (Doctoral dissertation, Concordia University)
Sghaier O, Amayri M, Bouguila N (2024) Libby-Novick Beta-Liouville Distribution for Enhanced Anomaly Detection in Proportional Data. ACM Trans Intell Syst Technol
Sghaier O, Amayri M, Bouguila N (2023) Multivariate Beta Normality Scores Approach for Deep Anomaly Detection in Images Using Transformations. In: 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp 3428–3433. IEEE
Funding
This work was funded via NSERC project number 6656-2017.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
Not applicable
Competing interests
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
When working with highly imbalanced datasets, precision-recall curves (PRC) can often provide more meaningful insights than the ROC curve. To ensure thoroughness, we present the performance of all baseline models in terms of the area under the precision-recall curve (AUPR). The following table summarizes the AUPR results for the three image datasets.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sghaier, O., Amayri, M. & Bouguila, N. Dirichlet and Liouville-based normality scores for deep anomaly detection using transformations: applications to images and beyond images. Appl Intell 55, 25 (2025). https://doi.org/10.1007/s10489-024-05892-2
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05892-2