Abstract
Generating pseudo-healthy reconstructions of images is an effective way to detect anomalies, as identifying the differences between the reconstruction and the original can localise arbitrary anomalies whilst also providing interpretability for an observer by displaying what the image ‘should’ look like. All existing reconstruction-based methods have a common shortcoming; they assume that models trained on purely normal data are incapable of reproducing pathologies yet also able to fully maintain healthy tissue. These implicit assumptions often fail, with models either not recovering normal regions or reproducing both the normal and abnormal features. We rectify this issue using image-conditioned diffusion models. Our model takes the input image as conditioning and is explicitly trained to correct synthetic anomalies introduced into healthy images, ensuring that it removes anomalies at test time. This conditioning allows the model to attend to the entire image without any loss of information, enabling it to replicate healthy regions with high fidelity. We evaluate our method across four datasets and define a new state-of-the-art performance for residual-based anomaly detection. Code is available at https://github.com/matt-baugh/img-cond-diffusion-model-ad.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4(1), 1–13 (2017)
Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., et al.: Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv:1811.02629 (2018)
Baugh, M., Tan, J., Müller, J.P., Dombrowski, M., Batten, J., Kainz, B.: Many tasks make light work: learning to localise medical anomalies from multiple synthetic tasks. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023, MICCAI 2023, LNCS, vol. 14220, pp. 162–172. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43907-0_16
Baugh, M., Tan, J., Vlontzos, A., Müller, J.P., Kainz, B.: nnOOD: a framework for benchmarking self-supervised anomaly localisation methods. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2022, LNCS, vol. 13563, pp. 103–112. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16749-2_10
Baur, C., Denner, S., Wiestler, B., Navab, N., Albarqouni, S.: Autoencoders for unsupervised anomaly segmentation in brain MR images: a comparative study. Med. Image Anal. 69, 101952 (2021)
Bercea, C.I., Rueckert, D., Schnabel, J.A.: What do AEs Learn? Challenging common assumptions in unsupervised anomaly detection. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023, MICCAI 2023, LNCS, vol. 14224, pp. 304–314. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43904-9_30
Bercea, C.I., Wiestler, B., Rueckert, D., Schnabel, J.A.: Reversing the abnormal: pseudo-healthy generative networks for anomaly detection. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023, MICCAI 2023, LNCS, vol. 14224, pp. 293–303. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43904-9_29
Chen, X., You, S., Tezcan, K.C., Konukoglu, E.: Unsupervised lesion detection via image restoration with a normative prior. Med. Image Anal. 64, 101713 (2020)
Defard, T., Setkov, A., Loesch, A., Audigier, R.: PaDiM: a patch distribution modeling framework for anomaly detection and localization. In: Del Bimbo, A., et al. (eds.) ICPR 2021. LNCS, vol. 12664, pp. 475–489. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68799-1_35
Deng, H., Li, X.: Anomaly detection via reverse distillation from one-class embedding. In: CVPR 2022. pp. 9737–9746 (2022)
Ghorbel, A., Aldahdooh, A., Albarqouni, S., Hamidouche, W.: Transformer based models for unsupervised anomaly segmentation in brain MR images. arXiv:2207.02059 (2022)
Graham, M.S., et al.: Unsupervised 3d out-of-distribution detection with latent diffusion models. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023, MICCAI 2023, LNCS, vol. 14220, pp 446–456. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43907-0_43
Gudovskiy, D., Ishizaka, S., Kozuka, K.: Cflow-ad: real-time unsupervised anomaly detection with localization via conditional normalizing flows. In: WACV, pp. 98–107 (2022)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. NeurIPS 2020 33, 6840–6851 (2020)
Iqbal, H., Khalid, U., Chen, C., Hua, J.: Unsupervised anomaly detection in medical images using masked diffusion model. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds.) Machine Learning in Medical Imaging, MLMI 2023, LNCS, vol. 14348, pp. 372–381. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-45673-2_37
Kascenas, A., Pugeault, N., O’Neil, A.Q.: Denoising autoencoders for unsupervised anomaly detection in brain MRI. In: MICCAI 2022, pp. 653–664. PMLR (2022)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: ICLR 2014 (2014)
Lagogiannis, I., Meissen, F., Kaissis, G., Rueckert, D.: Unsupervised pathology detection: a deep dive into the state of the art. IEEE Trans. Med. Imaging 43(1), 241–252 (2024)
Li, C.L., Sohn, K., Yoon, J., Pfister, T.: CutPaste: self-supervised learning for anomaly detection and localization. In: CVPR 2021, pp. 9664–9674 (2021)
Li, J., et al.: Fast non-markovian diffusion model for weakly supervised anomaly detection in brain MR images. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2023, MICCAI 2023, LNCS, vol. 14224, pp. 579–589. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43904-9_56
Li, T., Gao, Y., Wang, K., Guo, S., Liu, H., Kang, H.: Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf. Sci. 501, 511–522 (2019)
Liew, S.L., Anglin, J.M., Banks, N.W., Sondag, M., Ito, K.L., et al.: A large, open source dataset of stroke anatomical brain images and manual lesion segmentations. Sci. Data 5(1), 1–11 (2018)
Liu, W., Li, R., Zheng, M., Karanam, S., Wu, Z., et al.: Towards visually explaining variational autoencoders. In: CVPR 2020, pp. 8642–8651 (2020)
Marimont, S.N., Baugh, M., Siomos, V., Tzelepis, C., Kainz, B., Tarroni, G.: Disyre: diffusion-inspired synthetic restoration for unsupervised anomaly detection. arXiv preprint arXiv:2311.15453 (2023)
Meissen, F., Paetzold, J., Kaissis, G., Rueckert, D.: Unsupervised anomaly localization with structural feature-autoencoders. arXiv:2208.10992 (2022)
Meissen, F., Wiestler, B., Kaissis, G., Rueckert, D.: On the pitfalls of using the residual error as anomaly score. In: MIDL 2022, pp. 914–928. PMLR (2022)
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Müller, P.J., Baugh, M., Tan, J., Dombrowski, M., Kainz, B.: Confidence-aware and self-supervised image anomaly localisation. In: Sudre, C.H., Baumgartner, C.F., Dalca, A., Mehta, R., Qin, C., Wells, W.M. (eds.) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, UNSURE 2023, LNCS, vol. 14291, pp. 177–187. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-44336-7_18
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., et al.: Pytorch: an imperative style, high-performance deep learning library. In: NeurIPS 2019, vol. 32 (2019)
Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30–44 (2019)
Silva-Rodríguez, J., Naranjo, V., Dolz, J.: Constrained unsupervised anomaly segmentation. Med. Image Anal. 80, 102526 (2022)
Tan, J., Hou, B., Day, T., Simpson, J., Rueckert, D., Kainz, B.: Detecting outliers with poisson image interpolation. In: MICCAI 2021, pp. 581–591 (2021)
Taylor, J.R., Williams, N., Cusack, R., Auer, T., Shafto, M.A., et al.: The Cambridge Centre for ageing and neuroscience (Cam-CAN) data repository: structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample. Neuroimage 144, 262–269 (2017)
Tian, Y. et al.: Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds.) Machine Learning in Medical Imaging, MLMI 2023, LNCS, vol. 14349, pp. 11–21. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-45676-3_2
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Zimmerer, D., Full, P.M., Isensee, F., Jäger, P., Adler, T., et al.: Mood 2020: a public benchmark for out-of-distribution detection and localization on medical images. IEEE Trans. Med. Imaging 41(10), 2728–2738 (2022)
Zimmerer, D., Isensee, F., Petersen, J., Kohl, S., Maier-Hein, K.: Unsupervised anomaly localization using variational auto-encoders. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2019, MICCAI 2019, LNCS, vol. 11767, pp. 289–297. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_32
Acknowledgments
We received support by an EPSRC DTP award (MB), the ERC project MIA-NORMAL 101083647, the State of Bavaria (HTA), and DFG 512819079. HPC resources were provided by NHR@FAU of FAU Erlangen-Nürnberg under the NHR projects b180dc and b143dc. NHR funding is provided by federal and Bavarian state authorities. NHR@FAU hardware is partially funded by DFG - 440719683.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Baugh, M. et al. (2025). Image-Conditioned Diffusion Models for Medical Anomaly Detection. In: Sudre, C.H., Mehta, R., Ouyang, C., Qin, C., Rakic, M., Wells, W.M. (eds) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2024. Lecture Notes in Computer Science, vol 15167. Springer, Cham. https://doi.org/10.1007/978-3-031-73158-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-73158-7_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73157-0
Online ISBN: 978-3-031-73158-7
eBook Packages: Computer ScienceComputer Science (R0)
