Skip to main content
SpringerLink
Log in

Versatile anomaly detection method for medical images with semi-supervised flow-based generative models

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Radiologists interpret many medical images and clinical practice demands timely interpretation, resulting in a heavy workload. To reduce the workload, here we formulate and validate a method that can handle different types of medical image and can detect virtually all types of lesion in a medical image. For the first time, we show that two flow-based deep generative (FDG) models can predict the logarithm posterior probability in a semi-supervised approach.

Methods

We adopt two FDG models in conjunction with Bayes’ theorem to predict the logarithm posterior probability that a medical image is normal. We trained one of the FDG models with normal images and the other FDG model with normal and non-normal images.

Results

We validated the method using two types of medical image: chest X-ray images (CXRs) and brain computed tomography images (BCTs). The area under the receiver operating characteristic curve for pneumonia-like opacities in CXRs was 0.839 on average, and for infarction in BCTs was 0.904.

Conclusion

We formulated a method of predicting the logarithm posterior probability using two FDG models. We validated that the method can detect abnormal findings in CXRs and BCTs with both an acceptable performance for testing and a comparatively light workload for training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of data and material

The dataset of chest X-ray images is open to the public and known as the Radiological Society of North America (RSNA) Pneumonia Detection Challenge dataset. The dataset of brain-computed tomography images is protected under the laws of our institution; hence, it is not open to the public.

References

  1. Pang G, Shen C, Cao L, Hengel AVD (2020) Deep learning for anomaly detection: a review. arXiv preprint arXiv:2007.02500

  2. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  3. Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, Summers RM, Giger ML (2019) Deep learning in medical imaging and radiation therapy. Med Phys 46(1):e1–e36

    Article  Google Scholar 

  4. Kermany DS, Goldbaum M, Cai W, Valentim CC, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, Dong J, Prasadha MK, Pei J, Ting MY, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R, Zheng L, Hou R, Shi W, Fu X, Duan Y, Huu VA, Wen C, Zhang ED, Zhang CL, Li O, Wang X, Singer MA, Sun X, Xu J, Tafreshi A, Lewis MA, Xia H, Zhang K (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131

    Article  CAS  Google Scholar 

  5. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Patel BN, Yeom KW, Shpanskaya K, Blankenberg FG, Seekins J, Amrhein TJ, Mong DA, Halabi SS, Zucker EJ, Ng AY, Lungren MP (2018) Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med 15(11):e1002686

    Article  Google Scholar 

  6. Tang Y-X, Tang Y-B, Peng Y, Yan K, Bagheri M, Redd BA, Brandon CJ, Lu Z, Han M, Xiao J, Summers RM (2020) Automated abnormality classification of chest radiographs using deep convolutional neural networks. NPJ Digit Med 3(1):1–8

    Article  Google Scholar 

  7. Kuo W, Häne C, Mukherjee P, Malik J, Yuh EL (2019) Expert-level detection of acute intracranial hemorrhage on head computed tomography using deep learning. Proc Natl Acad Sci 116(45):22737–22745

    Article  CAS  Google Scholar 

  8. Prevedello LM, Erdal BS, Ryu JL, Little KJ, Demirer M, Qian S, White RD (2017) Automated critical test findings identification and online notification system using artificial intelligence in imaging. Radiology 285(3):923–931

    Article  Google Scholar 

  9. Titano JJ, Badgeley M, Schefflein J, Pain M, Su A, Cai M, Swinburne N, Zech J, Kim J, Bederson J, Mocco J, Drayer B, Lehar J, Cho S, Costa A, Oermann EK (2018) Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat Med 24(9):1337–1341

    Article  CAS  Google Scholar 

  10. Patel A, Van De Leemput SC, Prokop M, Van Ginneken B, Manniesing R (2019) Image level training and prediction: intracranial hemorrhage identification in 3D non-contrast CT. IEEE Access 7:92355–92364

    Article  Google Scholar 

  11. Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1\(\times \) 1 convolutions. In: Proceedings of the 32nd international conference on neural information processing systems, pp 10236–10245

  12. Shih G, Wu CC, Halabi SS, Kohli MD, Prevedello LM, Cook TS, Sharma A, Amorosa JK, Arteaga V, Galperin-Aizenberg M, Gill RR, Godoy MC, Hobbs S, Jeudy J, Laroia A, Shah PN, Vummidi D, Yaddanapudi K, Stein A (2019) Augmenting the national institutes of health chest radiograph dataset with expert annotations of possible pneumonia. Radiol Artif Intell 1(1):e180041

    Article  Google Scholar 

  13. Nakao T, Hanaoka S, Nomura Y, Murata M, Takenaga T, Miki S, Watadani T, Yoshikawa T, Hayashi N, Abe O (2021) Unsupervised deep anomaly detection in chest radiographs. J Digit Imag 34:1–10

    Article  Google Scholar 

  14. OpenAI (2018) Glow software. https://github.com/openai/glow/. Accessed 2 July 2021

  15. Youden WJ (1950) Index for rating diagnostic tests. Cancer 3(1):32–35

    Article  CAS  Google Scholar 

  16. Tang Y-X, Tang Y-B, Han M, Xiao J, Summers RM (2019) Abnormal chest X-ray identification with generative adversarial one-class classifier. In: 2019 IEEE 16th international symposium on biomedical imaging, pp 1358–1361

  17. Tang Y-X, Tang Y-B, Han M, Xiao J, Summers RM (2019) Deep adversarial one-class learning for normal and abnormal chest radiograph classification. In: Medical imaging 2019: computer-aided diagnosis, vol 10950, p 1095018. International Society for Optics and Photonics

  18. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chest X-ray 8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2097–2106

  19. Behrmann J, Grathwohl W, Chen RTQ, Duvenaud D, Jacobsen J-H (2019) Invertible residual networks. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of the 36th international conference on machine learning, vol 97, pp 573–582, PMLR, 09–15 Jun

  20. Chen RT, Behrmann J, Duvenaud DK, Jacobsen J-H (2019) Residual flows for invertible generative modeling. In: Advances in neural information processing systems, pp 9916–9926

  21. Ren J, Liu PJ, Fertig E, Snoek J, Poplin R, De Pristo MA, Dillon JV, Lakshminarayanan B (2019) Likelihood ratios for out-of-distribution detection. arXiv preprint arXiv:1906.02845

  22. Serrà J, Álvarez D, Gómez V, Slizovskaia O, Nú nez JF, Luque J (2019) Input complexity and out-of-distribution detection with likelihood-based generative models. arXiv preprint arXiv:1909.11480

  23. Schirrmeister RT, Zhou Y, Ball T, Zhang D (2020) Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. arXiv preprint arXiv:2006.10848

  24. Shibata H, Hanaoka S, Nomura Y, Nakao T, Sato I, Hayashi N, Abe O (2020) Anomaly detection in chest radiographs with a weakly supervised flow-based deep learning method. arXiv preprint arXiv:2001.07847v1

Download references

Funding

The Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, is sponsored by HIMEDIC Inc. and Siemens Healthcare K.K. This work was supported in part by JSPS Grants-in-Aid for Scientific Research (KAKENHI Grant Nos. 18K12095 and 18K12096). This work was also supported by the Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures and High Performance Computing Infrastructure projects in Japan (Project IDs: jh190047-DAH and jh200042-DAH).

Author information

Authors and Affiliations

  1. Department of Computational Diagnostic Radiology and Preventive Medicine, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan

    Hisaichi Shibata, Yukihiro Nomura, Takahiro Nakao & Naoto Hayashi

  2. Department of Radiology, The University of Tokyo Hospital, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan

    Shouhei Hanaoka, Issei Sato & Osamu Abe

  3. Department of Computer Science, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan

    Issei Sato

  4. Center for Advanced Intelligence Project, RIKEN, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan

    Issei Sato

  5. Division of Radiology and Biomedical Engineering, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan

    Daisuke Sato & Osamu Abe

Authors
  1. Hisaichi Shibata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shouhei Hanaoka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yukihiro Nomura
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Takahiro Nakao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Issei Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daisuke Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Naoto Hayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Osamu Abe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hisaichi Shibata.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethics approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1975 Helsinki Declaration, as revised in 2008(5).

Code availability

We slightly modified Glow software by Open AI, which is currently open to the public on GitHub (https://github.com/openai/glow), although our code is currently not open to the public.

Consent to participate

For this type of study, formal consent is not required.

Consent for publication

For this type of study, formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shibata, H., Hanaoka, S., Nomura, Y. et al. Versatile anomaly detection method for medical images with semi-supervised flow-based generative models. Int J CARS 16, 2261–2267 (2021). https://doi.org/10.1007/s11548-021-02480-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02480-4

Keywords

Search

Navigation