Skip to main content

Advertisement

Springer Nature Link
Log in

A survey on deep learning applied to medical images: from simple artificial neural networks to generative models

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep learning techniques, in particular generative models, have taken on great importance in medical image analysis. This paper surveys fundamental deep learning concepts related to medical image generation. It provides concise overviews of studies which use some of the latest state-of-the-art models from last years applied to medical images of different injured body areas or organs that have a disease associated with (e.g., brain tumor and COVID-19 lungs pneumonia). The motivation for this study is to offer a comprehensive overview of artificial neural networks (NNs) and deep generative models in medical imaging, so more groups and authors that are not familiar with deep learning take into consideration its use in medicine works. We review the use of generative models, such as generative adversarial networks and variational autoencoders, as techniques to achieve semantic segmentation, data augmentation, and better classification algorithms, among other purposes. In addition, a collection of widely used public medical datasets containing magnetic resonance (MR) images, computed tomography (CT) scans, and common pictures is presented. Finally, we feature a summary of the current state of generative models in medical image including key features, current challenges, and future research paths.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Akazawa M, Hashimoto K (2021) Artificial intelligence in gynecologic cancers: current status and future challenges - a systematic review. Artif Intell Med 120:102164. https://doi.org/10.1016/j.artmed.2021.102164

    Article  Google Scholar 

  2. de Siqueira VS, Borges MM, Furtado RG, Dourado CN, da Costa RM (2021) Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: a systematic review. Artif Intell Med 120:102165. https://doi.org/10.1016/j.artmed.2021.102165

    Article  Google Scholar 

  3. Fernando T, Gammulle H, Denman S, Sridharan S, Fookes C (2021) Deep learning for medical anomaly detection - a survey. ACM Comput Surv 54:7. https://doi.org/10.1145/3464423

    Article  Google Scholar 

  4. Chen J, Li K, Zhang Z, Li K, Yu PS (2021) A survey on applications of artificial intelligence in fighting against covid-19. ACM Comput Surv 54:8. https://doi.org/10.1145/3465398

    Article  Google Scholar 

  5. Sah M, Direkoglu C (2022) A survey of deep learning methods for multiple sclerosis identification using brain mri images. Neural Comput Appl 34(10):7349–7373. https://doi.org/10.1007/s00521-022-07099-3

    Article  Google Scholar 

  6. Abdou MA (2022) Literature review: efficient deep neural networks techniques for medical image analysis. Neural Comput Appl 34(8):5791–5812. https://doi.org/10.1007/s00521-022-06960-9

    Article  Google Scholar 

  7. Zhai J, Zhang S, Chen J, He Q (2018) Autoencoder and its various variants. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 415–419. IEEE, Miyazaki, Japan. https://doi.org/10.1109/SMC.2018.00080

  8. Kazeminia S, Baur C, Kuijper A, van Ginneken B, Navab N, Albarqouni S, Mukhopadhyay A (2020) Gans for medical image analysis. Artif Intell Med 109:101938. https://doi.org/10.1016/j.artmed.2020.101938

    Article  Google Scholar 

  9. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680

    Google Scholar 

  10. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/j.neunet.2014.09.003

    Article  Google Scholar 

  11. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  12. Clevert D-A, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network learning by exponential linear units (elus). In: 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings. ICLR, San Juan, Puerto Rico

  13. Jadon S (2020) A survey of loss functions for semantic segmentation. In: 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp. 1–7. IEEE, Región Metropolitana, Chile. https://doi.org/10.1109/cibcb48159.2020.9277638

  14. Lin T, Goyal P, Girshick RB, He K, Dollár P (2017) Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2999–3007. https://doi.org/10.1109/ICCV.2017.324

  15. Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, Québec City, Canada, pp 240–248

  16. Salehi SSM, Erdogmus D, Gholipour A (2017) Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: Wang Q, Shi Y, Suk H-I, Suzuki K (eds) Machine Learning Medcine in Imaging. Springer, Cham, pp 379–387

    Chapter  Google Scholar 

  17. Hayder Z, He X, Salzmann M (2017) Boundary-aware instance segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 587–595. https://doi.org/10.1109/CVPR.2017.70

  18. Taghanaki SA, Zheng Y, Zhou SK, Georgescu B, Sharma PS, Xu D, Comaniciu D, Hamarneh G (2019) Combo loss: handling input and output imbalance in multi-organ segmentation. Computerized Medi Imag Gr: Off J Computerized Med Imag Soc 75:24–33

    Article  Google Scholar 

  19. Abraham N, Khan NM (2019) A novel focal tversky loss function with improved attention u-net for lesion segmentation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 683–687. IEEE

  20. Berman M, Triki AR, Blaschko MB (2018) The lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4413–4421

  21. Bollschweiler EH, Möonig SP, Hensler K, Baldus SE, Maruyama K, Hölscher AH (2004) Artificial neural network for prediction of lymph node metastases in gastric cancer: a phase ii diagnostic study. Annal Surg Oncol 11:506–511. https://doi.org/10.1245/ASO.2004.04.018

    Article  Google Scholar 

  22. Dietzel M, Baltzer PAT, Dietzel A, Vag T, Gröschel T, Gajda M, Camara O, Kaiser WA (2010) Application of artificial neural networks for the prediction of lymph node metastases to the ipsilateral axilla - initial experience in 194 patients using magnetic resonance mammography. Acta Radiologica 51:851–858. https://doi.org/10.3109/02841851.2010.498444

    Article  Google Scholar 

  23. Biglarian A, Bakhshi E, Gohari MR, Khodabakhshi R (2012) Artificial neural network for prediction of distant metastasis in colorectal cancer. Asian Pacific J Cancer Prevent 13:927–930. https://doi.org/10.7314/APJCP.2012.13.3.927

    Article  Google Scholar 

  24. Gardner GG, Keating D, Williamson TH, Elliott AT (1996) Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool. Br J Ophthalmol 80:940–944. https://doi.org/10.1136/bjo.80.11.940

    Article  Google Scholar 

  25. Sinthanayothin C, Boyce JF, Cook HL, Williamson TH (1999) Automated localisation of the optic disc, fovea, and retinal blood vessels from digital colour fundus images. Br J Ophthalmol 83:902–910. https://doi.org/10.1136/bjo.83.8.902

    Article  Google Scholar 

  26. Özbay Y, Ceylan R, Karlik B (2006) A fuzzy clustering neural network architecture for classification of ecg arrhythmias. Computers Biol Med 36:376–388. https://doi.org/10.1016/j.compbiomed.2005.01.006

    Article  Google Scholar 

  27. Osowski S, Linh TH (2001) Ecg beat recognition using fuzzy hybrid neural network. IEEE Trans Biomed Eng 48:1265–1271. https://doi.org/10.1109/10.959322

    Article  Google Scholar 

  28. Ozbay Y, Karlik B (2001) A recognition of ecg arrhythmias using artificial neural networks. In: 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 2, pp. 1680–1683. IEEE, Istanbul, Turkey. https://doi.org/10.1109/IEMBS.2001.1020538

  29. Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: Dasgupta S, McAllester D (eds) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol 28. PMLR, Atlanta, Georgia, USA, pp 1139–1147

  30. Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12(61):2121–2159

    MathSciNet  MATH  Google Scholar 

  31. Tieleman T, Hinton G (2012) Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw Mach Learn 4:26–31

    Google Scholar 

  32. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

  33. Zeiler MD (2012) ADADELTA: an adaptive learning rate method. CoRR http://arxiv.org/abs/1212.5701

  34. Reddi SJ, Kale S, Kumar S (2018) On the convergence of adam and beyond. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, Vancouver, Canada

  35. Rahman T, Khandakar A, Qiblawey Y, Tahir A, Kiranyaz S, Abul Kashem SB, Islam MT, Al Maadeed S, Zughaier SM, Khan MS, Chowdhury MEH (2021) Exploring the effect of image enhancement techniques on covid-19 detection using chest x-ray images. Computers Biol Med 132:104319. https://doi.org/10.1016/j.compbiomed.2021.104319

    Article  Google Scholar 

  36. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  37. Clarke LP, Velthuizen RP, Phuphanich S, Schellenberg JD, Arrington JA, Silbiger M (1993) Mri: stability of three supervised segmentation techniques. Magnetic Resonan Imag 11:95–106. https://doi.org/10.1016/0730-725X(93)90417-C

    Article  Google Scholar 

  38. Veltri RW, Chaudhari M, Miller MC, Poole EC, O’Dowd GJ, Partin AW (2002) Comparison of logistic regression and neural net modeling for prediction of prostate cancer pathologic stage. Clin Chem 48:1828–1834. https://doi.org/10.1093/clinchem/48.10.1828

    Article  Google Scholar 

  39. Kan T, Shimada Y, Sato F, Ito T, Kondo K, Watanabe G, Maeda M, Yamasaki S, Meltzer SJ, Imamura M (2004) Prediction of lymph node metastasis with use of artificial neural networks based on gene expression profiles in esophageal squamous cell carcinoma. Annal Surg Oncol 11:1070. https://doi.org/10.1245/ASO.2004.03.007

    Article  Google Scholar 

  40. Nigam VP, Graupe D (2004) A neural-network-based detection of epilepsy. Neurol Res 26:55–60. https://doi.org/10.1179/016164104773026534

    Article  Google Scholar 

  41. Darby E, Nettimi T, Kodali S, Shih L (2005) Head and neck cancer metastasis prediction via artificial neural networks. In: 2005 IEEE Computational Systems Bioinformatics Conference - Workshops (CSBW’05), pp. 43–44. IEEE, Stanford, CA, USA. https://doi.org/10.1109/CSBW.2005.70

  42. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9. IEEE, Boston, MA, USA

  43. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  44. Dumoulin V, Visin F (2018) A guide to convolution arithmetic for deep learning. https://doi.org/10.48550/arXiv.1603.07285

  45. Shi W, Caballero J, Theis L, Huszar F, Aitken A, Ledig C, Wang Z (2016) Is the deconvolution layer the same as a convolutional layer?. https://doi.org/10.48550/arXiv.1609.07009

  46. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas, NV, US

  47. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826. IEEE, Las Vegas, NV, US

  48. Odena A, Dumoulin V, Olah C (2016) Deconvolution and checkerboard artifacts. Distill. https://doi.org/10.23915/distill.00003

  49. Nirthika R, Manivannan S, Ramanan A, Wang R (2022) Pooling in convolutional neural networks for medical image analysis: a survey and an empirical study. Neural Comput Appl 34(7):5321–5347. https://doi.org/10.1007/s00521-022-06953-8

    Article  Google Scholar 

  50. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36:193–202. https://doi.org/10.1007/BF00344251

    Article  MATH  Google Scholar 

  51. Lo S-CB, Lou S-LA, Lin J-S, Freedman MT, Chien MV, Mun SK (1995) Artificial convolution neural network techniques and applications for lung nodule detection. IEEE Trans Med Imag 14:711–718. https://doi.org/10.1109/42.476112

    Article  Google Scholar 

  52. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  53. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  54. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. ICLR, San Diego, California

  55. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)0.5MB model size. https://doi.org/10.48550/arXiv.1602.07360

  56. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243

  57. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-resnet and the impact of residual connections on learning. In: 31st AAAI Conference on Artificial Intelligence, AAAI 2017. AAAI press, San Francisco, California, pp 4278–4284

  58. Howard A, Sandler M, Chen B, Wang W, Chen L-C, Tan M, Chu G, Vasudevan V, Zhu Y, Pang R, Adam H, Le Q (2019) Searching for mobilenetv3. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, pp. 1314–1324. https://doi.org/10.1109/ICCV.2019.00140

  59. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pp. 234–241. Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28

  60. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) YOLOv4: Optimal Speed and Accuracy of Object Detection. https://doi.org/10.48550/arXiv.2004.10934

  61. Girshick R (2015) Fast r-cnn. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  62. Tao A, Barker J, Sarathy S (2016) DetectNet: Deep Neural Network for Object Detection in DIGITS. https://developer.nvidia.com/blog/detectnet-deep-neural-network-object-detection-digits/. Accesed 2021-10-26

  63. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. Lect Notes Computer Sci 9905:21–37. https://doi.org/10.1007/978-3-319-46448-0_2

    Article  Google Scholar 

  64. Brinker TJ et al (2019) Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer 113:47–54. https://doi.org/10.1016/j.ejca.2019.04.001

    Article  Google Scholar 

  65. Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis C-A, Gaiser T, Marx A, Valous NA, Ferber D, Jansen L, Reyes-Aldasoro CC, Zörnig I, Jäger D, Brenner H, Chang-Claude J, Hoffmeister M, Halama N (2019) Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLOS Med 16:1–22. https://doi.org/10.1371/journal.pmed.1002730

    Article  Google Scholar 

  66. Pereira RM, Bertolini D, Teixeira LO, Silla CN, Costa YMG (2020) Covid-19 identification in chest x-ray images on flat and hierarchical classification scenarios. Computer Methods Progr Biomed 194:105532. https://doi.org/10.1016/j.cmpb.2020.105532

    Article  Google Scholar 

  67. Yun J, Park J, Yu D, Yi J, Lee M, Park HJ, Lee J-G, Seo JB, Kim N (2019) Improvement of fully automated airway segmentation on volumetric computed tomographic images using a 2.5 dimensional convolutional neural net. Med Image Anal 51:13–20. https://doi.org/10.1016/j.media.2018.10.006

    Article  Google Scholar 

  68. Geng Y, Ren Y, Hou R, Han S, Rubin GD, Lo JY (2019) 2.5d cnn model for detecting lung disease using weak supervision. In: Hahn HK, Mori K (eds) Medical imaging 2019: computer-aided diagnosis, vol 10950. SPIE, San Diego, California, US, pp 924–928. https://doi.org/10.1117/12.2513631

    Chapter  Google Scholar 

  69. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D (2019) Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53:197–207. https://doi.org/10.1016/j.media.2019.01.012

    Article  Google Scholar 

  70. Nair T, Precup D, Arnold DL, Arbel T (2020) Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. Med Image Anal 59:101557. https://doi.org/10.1016/j.media.2019.101557

    Article  Google Scholar 

  71. Zhang J, Yu L, Chen D, Pan W, Shi C, Niu Y, Yao X, Xu X, Cheng Y (2021) Dense gan and multi-layer attention based lesion segmentation method for covid-19 ct images. Biomed Signal Process Control 69:102901. https://doi.org/10.1016/j.bspc.2021.102901

    Article  Google Scholar 

  72. Hesamian MH, Jia W, He X, Kennedy P (2019) Deep learning techniques for medical image segmentation: achievements and challenges. J Digital Imag 32:582–596. https://doi.org/10.1007/s10278-019-00227-x

    Article  Google Scholar 

  73. Zhang Y, Wu J, Liu Y, Chen Y, Chen W, Wu EX, Li C, Tang X (2021) A deep learning framework for pancreas segmentation with multi-atlas registration and 3d level-set. Med Image Anal 68:101884. https://doi.org/10.1016/j.media.2020.101884

    Article  Google Scholar 

  74. van den Oord A, Kalchbrenner N, Vinyals O, Espeholt L, Graves A, Kavukcuoglu K (2016) Conditional image generation with pixelcnn decoders. In: Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc., Barcelona, Spain

  75. Kingma DP, Dhariwal P (2018) Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems 31. https://doi.org/10.48550/arXiv.1807.03039

  76. Kingma D, Welling M (2014) Efficient gradient-based inference through transformations between bayes nets and neural nets. In: Proceedings of the 31st International Conference on Machine Learning, vol. 32, pp. 1782–1790. PMLR, Beijing, China

  77. Rezende DJ, Mohamed S, Wierstra D (2014) Stochastic backpropagation and approximate inference in deep generative models. In: Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 1278–1286. PMLR, Bejing, China

  78. Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348. https://doi.org/10.1016/j.neucom.2019.02.003

    Article  Google Scholar 

  79. Minaee S, Boykov Y, Porikli F, Plaza A, Kehtarnavaz N, Terzopoulos D (2022) Image segmentation using deep learning: a survey. IEEE Trans Pattern Anal Mach Intell 44(7):3523–3542. https://doi.org/10.1109/TPAMI.2021.3059968

    Article  Google Scholar 

  80. Asgari Taghanaki S, Abhishek K, Cohen JP, Cohen-Adad J, Hamarneh G (2021) Deep semantic segmentation of natural and medical images: a review. Artif Intell Rev 54(1):137–178. https://doi.org/10.1007/s10462-020-09854-1

    Article  Google Scholar 

  81. Ben-Cohen A, Diamant I, Klang E, Amitai M, Greenspan H (2016) Fully convolutional network for liver segmentation and lesions detection. In: Deep Learning and Data Labeling for Medical Applications. Springer, Cham, pp 77–85

  82. Li D, Yang J, Kreis K, Torralba A, Fidler S (2021) Semantic segmentation with generative models: Semi-supervised learning and strong out-of-domain generalization. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8296–8307. https://doi.org/10.1109/CVPR46437.2021.00820

  83. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, Canada, pp. 4396–4405. https://doi.org/10.1109/CVPR.2019.00453

  84. Imtiaz R, Khan TM, Naqvi SS, Arsalan M, Nawaz SJ (2021) Screening of glaucoma disease from retinal vessel images using semantic segmentation. Computers Electr Eng 91:107036. https://doi.org/10.1016/j.compeleceng.2021.107036

    Article  Google Scholar 

  85. Rehman MU, Cho S, Kim J, Chong KT (2021) Brainseg-net: Brain tumor mr image segmentation via enhanced encoder-decoder network. Diagnostics 11:2. https://doi.org/10.3390/diagnostics11020169

    Article  Google Scholar 

  86. Zunair H, Ben Hamza A (2021) Sharp u-net: Depthwise convolutional network for biomedical image segmentation. Computers Biol Med 136:104699. https://doi.org/10.1016/j.compbiomed.2021.104699

    Article  Google Scholar 

  87. Su R, Zhang D, Liu J, Cheng C (2021) Msu-net: Multi-scale u-net for 2d medical image segmentation. Front Genet 12:58. https://doi.org/10.3389/fgene.2021.639930

    Article  Google Scholar 

  88. Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH (2021) nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2):203–211. https://doi.org/10.1038/s41592-020-01008-z

    Article  Google Scholar 

  89. Zuo Q, Chen S, Wang Z (2021) R2au-net: Attention recurrent residual convolutional neural network for multimodal medical image segmentation. Security Commun Netw 2021:6625688. https://doi.org/10.1155/2021/6625688

    Article  Google Scholar 

  90. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser LU, Polosukhin I (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates Inc, Long Beach, California. https://doi.org/10.48550/arXiv.1706.03762

    Chapter  Google Scholar 

  91. Mnih V, Heess N, Graves A (2014) Recurrent models of visual attention. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. Curran Associates Inc, Quebec, Canada, pp 2204–2212

    Google Scholar 

  92. Ouyang X, Huo J, Xia L, Shan F, Liu J, Mo Z, Yan F, Ding Z, Yang Q, Song B, Shi F, Yuan H, Wei Y, Cao X, Gao Y, Wu D, Wang Q, Shen D (2020) Dual-sampling attention network for diagnosis of covid-19 from community acquired pneumonia. IEEE Trans Med Imag 39(8):2595–2605. https://doi.org/10.1109/TMI.2020.2995508

    Article  Google Scholar 

  93. Pang S, Du A, Orgun MA, Wang Y, Yu Z (2021) Tumor attention networks: Better feature selection, better tumor segmentation. Neural Netw 140:203–222. https://doi.org/10.1016/j.neunet.2021.03.006

    Article  Google Scholar 

  94. Sinha A, Dolz J (2021) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inf 25(1):121–130. https://doi.org/10.1109/JBHI.2020.2986926

    Article  Google Scholar 

  95. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X, Chen X (2016) Improved techniques for training gans. In: Lee D, Sugiyama M, Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems, vol 29. Curran Associates Inc, Barcelona, Spain, pp 2234–2242

    Google Scholar 

  96. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. NIPS’17, pp. 6629–6640. Curran Associates Inc., Red Hook, NY, USA

  97. Mirza M, Osindero S (2014) Conditional Generative Adversarial Nets. https://doi.org/10.48550/arXiv.1411.1784

  98. Lee M, Seok J (2019) Controllable generative adversarial network. IEEE. Access 7:28158–28169. https://doi.org/10.1109/ACCESS.2019.2899108

    Article  Google Scholar 

  99. Ghassemi N, Shoeibi A, Rouhani M (2020) Deep neural network with generative adversarial networks pre-training for brain tumor classification based on mr images. Biomed Signal Process Control 57:101678. https://doi.org/10.1016/j.bspc.2019.101678

    Article  Google Scholar 

  100. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Precup D, Teh YW (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 214–223. PMLR, Sydney, Australia

  101. Shen Y, Gu J, Tang X, Zhou B (2020) Interpreting the latent space of gans for semantic face editing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, Washington, pp. 9240–9249. https://doi.org/10.1109/CVPR42600.2020.00926

  102. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, p. 149803. ICLR, San Juan, Puerto Rico

  103. Saito M, Matsumoto E, Saito S (2017) Temporal generative adversarial nets with singular value clipping. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2849–2858. https://doi.org/10.1109/ICCV.2017.308

  104. Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of gans for improved quality, stability, and variation. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, p. 149806. ICLR, Vancouver, Canada

  105. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, pp. 5967–5976. https://doi.org/10.1109/CVPR.2017.632

  106. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2242–2251. https://doi.org/10.1109/ICCV.2017.244

  107. Esser P, Rombach R, Ommer B (2021) Taming transformers for high-resolution image synthesis. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12868–12878. https://doi.org/10.1109/CVPR46437.2021.01268

  108. Brock A, Donahue J, Simonyan K (2019) Large scale gan training for high fidelity natural image synthesis. In: 7th International Conference on Learning Representation 2019. ICLR, New Orleans

  109. Nema S, Dudhane A, Murala S, Naidu S (2020) Rescuenet: an unpaired gan for brain tumor segmentation. Biomed Signal Process Control 55:101641. https://doi.org/10.1016/j.bspc.2019.101641

    Article  Google Scholar 

  110. Klages P, Benslimane I, Riyahi S, Jiang J, Hunt M, Deasy JO, Veeraraghavan H, Tyagi N (2020) Patch-based generative adversarial neural network models for head and neck mr-only planning. Med Phys 47:626–642. https://doi.org/10.1002/mp.13927

    Article  Google Scholar 

  111. Do W, Seo S, Han Y, Ye JC, Choi SH, Park S (2020) Reconstruction of multicontrast mr images through deep learning. Med Phys 47:983–997. https://doi.org/10.1002/mp.14006

    Article  Google Scholar 

  112. Carreras-Delgado JL, Pérez-Dueñas V, Riola-Parada C, García-Cañamaque L (2016) Pet/mri: A luxury or a necessity? Revista Española de Medicina Nuclear e Imagen Molecular (English Edition) 35:313–320. https://doi.org/10.1016/j.remnie.2016.07.002

    Article  Google Scholar 

  113. Pozaruk A, Pawar K, Li S, Carey A, Cheng J, Sudarshan VP, Cholewa M, Grummet J, Chen Z, Egan G (2021) Augmented deep learning model for improved quantitative accuracy of mr-based pet attenuation correction in psma pet-mri prostate imaging. Eur J Nucl Med Mol Imag 48:9–20. https://doi.org/10.1007/s00259-020-04816-9

    Article  Google Scholar 

  114. Zhou X, Qiu S, Joshi PS, Xue C, Killiany RJ, Mian AZ, Chin SP, Au R, Kolachalama VB (2021) Enhancing magnetic resonance imaging-driven alzheimer’s disease classification performance using generative adversarial learning. Alzheimer’s Res Ther 13:60. https://doi.org/10.1186/s13195-021-00797-5

    Article  Google Scholar 

  115. Lei B, Xia Z, Jiang F, Jiang X, Ge Z, Xu Y, Qin J, Chen S, Wang T, Wang S (2020) Skin lesion segmentation via generative adversarial networks with dual discriminators. Med Image Anal 64:101716. https://doi.org/10.1016/j.media.2020.101716

    Article  Google Scholar 

  116. Qin Z, Liu Z, Zhu P, Xue Y (2020) A gan-based image synthesis method for skin lesion classification. Computer Methods Progr Biomed 195:105568. https://doi.org/10.1016/j.cmpb.2020.105568

    Article  Google Scholar 

  117. Rasheed J, Hameed AA, Djeddi C, Jamil A, Al-Turjman F (2021) A machine learning-based framework for diagnosis of covid-19 from chest x-ray images. Interdisciplinary Sci: Comput Life Sci 13:103–117. https://doi.org/10.1007/s12539-020-00403-6

    Article  Google Scholar 

  118. Albahli S (2021) A deep neural network to distinguish covid-19 from other chest diseases using x-ray images. Curr Med Imag Formerly Curr Med Imag Rev 17:109–119. https://doi.org/10.2174/1573405616666200604163954

    Article  Google Scholar 

  119. Li Z, Zhang J, Li B, Gu X, Luo X (2021) Covid-19 diagnosis on ct scan images using a generative adversarial network and concatenated feature pyramid network with an attention mechanism. Med Phys 48:4334–4349. https://doi.org/10.1002/mp.15044

    Article  Google Scholar 

  120. Pang T, Wong JHD, Ng WL, Chan CS (2021) Semi-supervised gan-based radiomics model for data augmentation in breast ultrasound mass classification. Computer Methods Progr Biomed 203:106018. https://doi.org/10.1016/j.cmpb.2021.106018

    Article  Google Scholar 

  121. Davidson TR, Falorsi L, Cao ND, Kipf T, Tomczak JM (2018) Hyperspherical variational auto-encoders. In: 34th Conference on Uncertainty in Artificial Intelligence 2018, vol. 2, pp. 856–865. Association For Uncertainty in Artificial Intelligence, Monterey, California

  122. Kingma DP, Welling M (2019) An introduction to variational autoencoders. Foundations Trends Mach Learn 12(4):307–392. https://doi.org/10.1561/2200000056

    Article  MATH  Google Scholar 

  123. Uzunova H, Schultz S, Handels H, Ehrhardt J (2019) Unsupervised pathology detection in medical images using conditional variational autoencoders. Int J Computer Assist Radiol Surg 14:451–461. https://doi.org/10.1007/s11548-018-1898-0

    Article  Google Scholar 

  124. Sohn K, Lee H, Yan X (2015) Learning structured output representation using deep conditional generative models. Adv Neural Inf Process Syst 28:3483–3491

    Google Scholar 

  125. Akrami H, Joshi AA, Li J, Aydore S, Leahy RM (2020) Brain lesion detection using a robust variational autoencoder and transfer learning. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 786–790. IEEE, Iowa City, IA, USA. https://doi.org/10.1109/ISBI45749.2020.9098405

  126. Marimont SN, Tarroni G (2021) Anomaly detection through latent space restoration using vector quantized variational autoencoders. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1764–1767. IEEE, Nice, Italy. https://doi.org/10.1109/ISBI48211.2021.9433778

  127. Wei L, Owen D, Rosen B, Guo X, Cuneo K, Lawrence TS, Haken RT, Naqa IE (2021) A deep survival interpretable radiomics model of hepatocellular carcinoma patients. Phys Medica 82:295–305. https://doi.org/10.1016/j.ejmp.2021.02.013

    Article  Google Scholar 

  128. Kou W, Carlson DA, Baumann AJ, Donnan E, Luo Y, Pandolfino JE, Etemadi M (2021) A deep-learning-based unsupervised model on esophageal manometry using variational autoencoder. Artif Intell Med 112:102006. https://doi.org/10.1016/j.artmed.2020.102006

    Article  Google Scholar 

  129. Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: Balcan MF, Weinberger KQ (eds.), Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 48, pp. 1558–1566. PMLR, New York, New York, USA

  130. Bao J, Chen D, Wen F, Li H, Hua G (2017) Cvae-gan: Fine-grained image generation through asymmetric training. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2764–2773. IEEE, Venice, Italy. https://doi.org/10.1109/ICCV.2017.299

  131. Nakao T, Hanaoka S, Nomura Y, Murata M, Takenaga T, Miki S, Watadani T, Yoshikawa T, Hayashi N, Abe O (2021) Unsupervised deep anomaly detection in chest radiographs. J Digital Imag 34:418–427. https://doi.org/10.1007/s10278-020-00413-2

    Article  Google Scholar 

  132. Nguyen A, Clune J, Bengio Y, Dosovitskiy A, Yosinski J (2017) Plug & play generative networks: Conditional iterative generation of images in latent space, Honolulu, Hawaii, pp. 3510–3520. https://doi.org/10.1109/CVPR.2017.374

  133. Baur C, Denner S, Wiestler B, Navab N, Albarqouni S (2021) Autoencoders for unsupervised anomaly segmentation in brain mr images: a comparative study. Med Image Anal 69:101952. https://doi.org/10.1016/j.media.2020.101952

    Article  Google Scholar 

  134. van den Oord A, Vinyals O, Kavukcuoglu K (2017) Neural discrete representation learning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS’17, pp. 6309–6318. Curran Associates Inc., Red Hook, NY, USA

  135. Donahue J, Krähenbühl P, Darrell T (2017) Adversarial Feature Learning. https://doi.org/10.48550/arXiv.1605.09782

  136. Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, Jack CR, Jagust WJ, Shaw LM, Toga AW, Trojanowski JQ, Weiner MW (2010) Alzheimer’s disease neuroimaging initiative (adni): clinical characterization. Neurology 74(3):201–209. https://doi.org/10.1212/WNL.0b013e3181cb3e25

    Article  Google Scholar 

  137. Ramzan F, Khan MUG, Rehmat A, Iqbal S, Saba T, Rehman A, Mehmood Z (2020) A deep learning approach for automated diagnosis and multi-class classification of alzheimer’s disease stages using resting-state fmri and residual neural networks. J Med Syst 44:37. https://doi.org/10.1007/s10916-019-1475-2

    Article  Google Scholar 

  138. Puente-Castro A, Fernandez-Blanco E, Pazos A, Munteanu CR (2020) Automatic assessment of alzheimer’s disease diagnosis based on deep learning techniques. Computers Biol Med 120:103764. https://doi.org/10.1016/j.compbiomed.2020.103764

    Article  Google Scholar 

  139. LaMontagne PJ, Benzinger TL, Morris JC, Keefe S, Hornbeck R, Xiong C, Grant E, Hassenstab J, Moulder K, Vlassenko AG, Raichle ME, Cruchaga C, Marcus D (2019) OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease. Cold Spring Harbor Laboratory Press. https://doi.org/10.1101/2019.12.13.19014902

  140. Cheng J, Huang W, Cao S, Yang R, Yang W, Yun Z, Wang Z, Feng Q (2015) Enhanced performance of brain tumor classification via tumor region augmentation and partition. PLOS ONE 10(10):1–13. https://doi.org/10.1371/journal.pone.0140381

    Article  Google Scholar 

  141. Deepak S, Ameer PM (2019) Brain tumor classification using deep cnn features via transfer learning. Computers Biol Med 111:103345. https://doi.org/10.1016/j.compbiomed.2019.103345

    Article  Google Scholar 

  142. Menze BH et al (2015) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imag 34(10):1993–2024. https://doi.org/10.1109/TMI.2014.2377694

    Article  Google Scholar 

  143. Zimmerer D, Petersen J, Köhler G, Jäger P, Full P, Roß T, Adler T, Reinke A, Maier-Hein L, Maier-Hein K (2021) Medical out-of-distribution analysis challenge 2021. Zenodo. https://doi.org/10.5281/zenodo.4573948

  144. Bándi P et al (2019) From detection of individual metastases to classification of lymph node status at the patient level: The camelyon17 challenge. IEEE Trans Med Imag 38(2):550–560. https://doi.org/10.1109/TMI.2018.2867350

    Article  Google Scholar 

  145. Heath M, Bowyer K, Kopans D, Moore R, Kegelmeyer P (2000) The digital database for screening mammography. Proceedings of the Fourth International Workshop on Digital Mammography 13. https://doi.org/10.1007/978-94-011-5318-8_75

  146. Agarwal R, Díaz O, Yap MH, Lladó X, Martí R (2020) Deep learning for mass detection in full field digital mammograms. Computers Biol Med 121:103774. https://doi.org/10.1016/j.compbiomed.2020.103774

    Article  Google Scholar 

  147. Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast. Acad Radiol 19:236–248. https://doi.org/10.1016/j.acra.2011.09.014

    Article  Google Scholar 

  148. Buda M, Saha A, Walsh R, Ghate S, Li N, Swiecicki A, Lo JY, Yang J, Mazurowski MA (2020) Data from the breast cancer screening – digital breast tomosynthesis (bcs-dbt). https://doi.org/10.7937/e4wt-cd02

  149. Nogay H, Akinci TC, Yilmaz M (2021) Comparative experimental investigation and application of five classic pre-trained deep convolutional neural networks via transfer learning for diagnosis of breast cancer. Adv Sci Technol Res J 15:1–8. https://doi.org/10.12913/22998624/137964

    Article  Google Scholar 

  150. Kermany DS et al (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172:1122–1131. https://doi.org/10.1016/j.cell.2018.02.010

    Article  Google Scholar 

  151. Khan AI, Shah JL, Bhat MM (2020) Coronet: a deep neural network for detection and diagnosis of covid-19 from chest x-ray images. Computer Methods Progr Biomed 196:105581. https://doi.org/10.1016/j.cmpb.2020.105581

    Article  Google Scholar 

  152. Minaee S, Kafieh R, Sonka M, Yazdani S, Soufi GJ (2020) Deep-covid: predicting covid-19 from chest x-ray images using deep transfer learning. Med Image Anal 65:101794. https://doi.org/10.1016/j.media.2020.101794

    Article  Google Scholar 

  153. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3462–3471. IEEE, Honolulu, HI, US. https://doi.org/10.1109/CVPR.2017.369

  154. Cohen JP, Morrison P, Dao L (2020) COVID-19 image data collection. https://doi.org/10.48550/ARXIV.2003.11597

  155. Armato SG III et al (2011) The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Med Phys 38(2):915–931. https://doi.org/10.1118/1.3528204

    Article  Google Scholar 

  156. Saltz J, Saltz M, Prasanna P, Moffitt R, Hajagos J, Bremer E, Balsamo J, Kurc T (2021) Stony Brook University COVID-19 Positive Cases [Data set]. https://doi.org/10.7937/TCIA.BBAG-2923

  157. ...Rotemberg V, Kurtansky N, Betz-Stablein B, Caffery L, Chousakos E, Codella N, Combalia M, Dusza S, Guitera P, Gutman D, Halpern A, Helba B, Kittler H, Kose K, Langer S, Lioprys K, Malvehy J, Musthaq S, Nanda J, Reiter O, Shih G, Stratigos A, Tschandl P, Weber J, Soyer HP (2021) A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientif Data 8:34. https://doi.org/10.1038/s41597-021-00815-z

    Article  Google Scholar 

  158. Roth HR, Farag A, Turkbey EB, Lu L, Liu J, Summers RM (2016). Data From Pancreas-CT. https://doi.org/10.7937/K9/TCIA.2016.tNB1kqBU

  159. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3730–3738. IEEE, Santiago, Chile. https://doi.org/10.1109/ICCV.2015.425

  160. Krizhevsky A, Hinton G, et al (2009) Learning multiple layers of features from tiny images, 32–33

  161. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213–3223. IEEE, Las Vegas, NV, USA

  162. Lin T-Y, Maire M, Belongie S, Bourdev L, Girshick R, Hays J, Perona P, Ramanan D, Zitnick CL, Dollár P (2014) Microsoft coco: Common objects in context. In: Computer Vision - ECCV 2014. Springer, Cham, pp 740–755

  163. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology

  164. Ebner NC, Riediger M, Lindenberger U (2010) Faces–a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav Res Methods 42:351–362. https://doi.org/10.3758/BRM.42.1.351

    Article  Google Scholar 

  165. Huiskes MJ, Lew MS (2008) The mir flickr retrieval evaluation. In: MIR ’08: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. MIR ’08, pp. 39–43. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1460096.1460104

  166. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Miami, FL, US, pp 248–255

  167. Huang GB, Ramesh M, Berg T, Learned-Miller E (October 2007) Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst

  168. Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J (2016) LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. https://doi.org/10.48550/arXiv.1506.03365

  169. LeCun Y, Cortes C (2010) MNIST handwritten digit database

  170. Everingham M, Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Kluwer Academic Publishers, USA. https://doi.org/10.1007/s11263-009-0275-4

    Book  Google Scholar 

  171. Apostolopoulos ID, Mpesiana TA (2020) Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Phys Eng Sci Med 43:635–640. https://doi.org/10.1007/s13246-020-00865-4

    Article  Google Scholar 

  172. Yap MH, Goyal M, Osman F, Martí R, Denton E, Juette A, Zwiggelaar R (2020) Breast ultrasound region of interest detection and lesion localisation. Artif Intell Med 107:101880. https://doi.org/10.1016/j.artmed.2020.101880

    Article  Google Scholar 

  173. Papanastasopoulos Z, Samala RK, Chan H-P, Hadjiiski L, Paramagul C, Helvie MA, Neal CH (2020) Explainable ai for medical imaging: deep-learning cnn ensemble for classification of estrogen receptor status from breast mri. In: Hahn HK, Mazurowski MA (eds) Medical imaging 2020: computer-aided diagnosis, vol 11314. SPIE, Houston, Texas, US, pp 228–235. https://doi.org/10.1117/12.2549298

    Chapter  Google Scholar 

  174. Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, Tang Y, Xiao A, Xu C, Xu Y, Yang Z, Zhang Y, Tao D (2022) A survey on vision transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence 1–1. https://doi.org/10.1109/TPAMI.2022.3152247

  175. Wu H, Chen S, Chen G, Wang W, Lei B, Wen Z (2022) Fat-net: feature adaptive transformers for automated skin lesion segmentation. Med Image Anal 76:102327. https://doi.org/10.1016/j.media.2021.102327

    Article  Google Scholar 

  176. Korkmaz Y, Dar SUH, Yurt M, Özbey M, Çukur T (2022) Unsupervised mri reconstruction via zero-shot learned adversarial transformers. IEEE Trans Med Imag 41(7):1747–1763. https://doi.org/10.1109/TMI.2022.3147426

    Article  Google Scholar 

  177. Song Y, Ermon S (2019) Generative modeling by estimating gradients of the data distribution. In: Wallach H, Larochelle H, Beygelzimer A, d’ Alché-Buc F, Fox E, Garnett R (eds) Advances in neural information processing systems. Curran Associates Inc, Vancouver, Canada

    Google Scholar 

  178. Jalal A, Arvinte M, Daras G, Price E, Dimakis AG, Tamir J (2021) Robust compressed sensing mri with deep generative priors. In: Advances in Neural Information Processing Systems, vol. 34, pp. 14938–14954. Curran Associates, Inc., Virtual Conference

  179. Chung H, Ye JC (2022) Score-based diffusion models for accelerated mri. Med Image Anal 80:102479. https://doi.org/10.1016/j.media.2022.102479

    Article  Google Scholar 

  180. Wang L, Liu Y, Wu R, Liu Y, Yan R, Ren S, Gui Z (2022) Image processing for low-dose ct via novel anisotropic fourth-order diffusion model. IEEE Access 10:50114–50124. https://doi.org/10.1109/ACCESS.2022.3172975

    Article  Google Scholar 

  181. Gomez T, Feyeux M, Boulant J, Normand N, David L, Paul-Gilloteaux P, Fréour T, Mouchère H (2022) A time-lapse embryo dataset for morphokinetic parameter prediction. Data in Brief 42:108258. https://doi.org/10.1016/j.dib.2022.108258

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported by MCIN/AEI/ 10.13039/501100011033 under the scope of the CURMIS4th project (Grant PID2020–113673RB-I00), the Consellería de Educación, Universidades e Formación Profesional (Xunta de Galicia) under the scope of the strategic funding of ED431C2018/55-GRC Competitive Reference Group, the “Centro singular de investigación de Galicia” (accreditation 2019–2022), and the European Union (European Regional Development Fund - ERDF)- Ref. ED431G2019/06. SING group thanks CITI (Centro de Investigación, Transferencia e Innovación) from the University of Vigo for hosting its IT infrastructure.

Funding

Pedro Celard is supported by a pre-doctoral fellowship from Xunta de Galicia (ED481A 2021/286). The funders had no role in study design, data collection and analysis, the decision to publish, or the preparation of the manuscript.

Author information

Authors and Affiliations

  1. Computer Science Department, Universidade de Vigo, Escuela Superior de Ingeniería Informática, Campus Universitario As Lagoas, 32004, Ourense, Spain

    P. Celard, E. L. Iglesias, J. M. Sorribes-Fdez, R. Romero, A. Seara Vieira & L. Borrajo

  2. CINBIO - Biomedical Research Centre, Universidade de Vigo, Campus Universitario Lagoas-Marcosende, 36310, Vigo, Spain

    P. Celard, E. L. Iglesias, J. M. Sorribes-Fdez, R. Romero, A. Seara Vieira & L. Borrajo

  3. SING Research Group, Galicia Sur Health Research Institute (IIS Galicia Sur), SERGAS-UVIGO, Vigo, Spain

    P. Celard, E. L. Iglesias, J. M. Sorribes-Fdez, R. Romero, A. Seara Vieira & L. Borrajo

Authors
  1. P. Celard
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. E. L. Iglesias
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. J. M. Sorribes-Fdez
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. R. Romero
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. A. Seara Vieira
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. L. Borrajo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to P. Celard.

Ethics declarations

Conflict of interest

Authors declare that there is not conflicts of interest.

Consent to participate

Authors have consented to the submission of this survey to the journal.

Consent for publication

Authors have consented to the publication of this survey to the journal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Celard, P., Iglesias, E.L., Sorribes-Fdez, J.M. et al. A survey on deep learning applied to medical images: from simple artificial neural networks to generative models. Neural Comput & Applic 35, 2291–2323 (2023). https://doi.org/10.1007/s00521-022-07953-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07953-4

Keywords