Full length articleA survey on deep learning in medicine: Why, how and when?
Introduction
In the coming years, Artificial Intelligence (AI) will have an increasingly important role in the field of medicine, where it is already making a difference today. Medicine based on the observation of events has been, for many years, ever since the time of Hippocrates, the epistemological guiding criterion of the healthcare profession. This approach has evolved, with the progress of medicine, into what is termed Evidence-Based Medicine (EBM). Today, indeed, medicine based on signs which cannot be observed by any human doctor but can become evident with the use of Big Data and Deep Learning (DL) techniques has been developed. Such techniques are able to consider and process much more information than is possible for any human. State-of-the-art Deep Neural Networks (DNNs), also known as DL modes [1] have demonstrated remarkable results in image processing, classification and data analysis.
DL is increasingly attracting the interest of researchers in the medical and healthcare sectors, since, by using medical data, it is possible to increase the accuracy of medical applications. In particular, DL is rapidly replacing classic neural network techniques, named artificial neural networks (ANNs), whose goal is to mimic the human brain. This trend can be motivated by the following reasons. Firstly, DL can provide a better interpretation of a very complex phenomenon than classic statistical approaches if high-dimensional datasets are available, and the performance of a DL is directly proportional to the input size. This is a common scenario in medicine, where, as pointed out in [2], large amounts of data (about 15 to 20 TB) are collected and stored in optimized databases every day, also by using Cloud computing platforms [3]. Furthermore, DL is characterized by a high degree of flexibility. Medical data include different types of unstructured data, such as images, signals, genetic expressions and text data. Thanks to the complexity of their architectures, DL frameworks are able to benefit from this heterogeneity by achieving high levels of abstraction in data analysis. Finally, the high level of automation [4]. ML algorithms require a manual intervention to select the fundamental information from the input data and the corresponding transformation rules [5]. This is a crucial challenge because an experts’ decision is needed and, therefore, there is a corresponding increase in the time and costs for a diagnosis [6]. However, DL can determine these elements by using large samples of examples. There are two main consequences of this facility. Firstly, there is a significant reduction in the cost and time of treatment and diagnosis, Secondly, the independence of the diagnosis means that patients can talk directly to data scientists and, by running software, can understand the cause of their disease and obtain the best treatment.
The application of AI in real contexts can result in numerous potential advantages, such as the execution speed, potential reduction in costs, both direct and indirect, better diagnostic accuracy, greater clinical and operational efficiency (“algorithms don’t sleep”) and the possibility of providing access to the clinical information even for people who cannot otherwise benefit from this for geographical, political and economic reasons.
A great number of publications and surveys have addressed the use of DL in medicine, focusing on specific challenges or medical fields [7], [8], [9], [10]. Nevertheless, most of these works are lacking in details, difficult to compare and do not provide the reader with a comprehensive overview of the applications of DL in the general medical area. Fig. 1 presents a keywords-generated tree-map extracted from a Scopus1 dataset composed of papers related to “Deep Learning” and “Medicine” as input words. From 2016 until now, more than 1200 papers have been considered within this dataset. The tree-map has been generated by using the bibliometrix R-package,2 an open-source tool for quantitative research in scientometrics and bibliometrics that includes all the main bibliometric methods of analysis. By analysing the tree-map (starting from the left side and considering the biggest squares), it is evident that DL in Medicine is mostly applied in the task of image processing, with a great focus on diagnostics. By continuing the analysis towards the right, some crucial keywords can be observed, such as “aged”, “personalized medicine”, and “classification”. Summarizing the results of the keyword-based tree-map, it is possible to have an overview of the main medical fields, the principal tasks performed, and the most frequently used algorithms relating to DL in medicine during the last few years.
Starting from the above considerations, in this paper our aim is to provide an extensive analysis of DL applications in medicine, also categorizing DL models in relation to their applications in different medical areas. Afterwards, a comprehensive study of the state-of-the-art DL in medicine will be performed, taking into account existing DL surveys focused on specific medical fields. In summary, with this paper we aim to make the following contribution:
- 1.
We will review the state-of-the-art in papers and surveys, especially of recent years, focusing on the application of DL in medicine, including all medical areas.
- 2.
We will present a categorization of the DL models used and applied in medicine and give clear definitions of each, also providing an overview of hybrid DL architectures.
- 3.
We will comprehensively classify medicine-related DL applications into macro-areas, also describing their sub-areas and the key aspects of the applied DL models.
- 4.
We will analyse and discuss the recent and open challenges related to DL in medicine, also addressing future research directions, in order to provide the reader with a clear overview of the real-world scenario.
The rest of the paper is organized as follows, as depicted in Fig. 2. In Section 2, the various DL models are described and analysed in depth, with hybrid architectures also presented. In Section 3, a comprehensive overview and classification of DL applications in medicine is provided, including also a description of the main properties of the applied DL models. Section 4 presents a review of the kinds of medical data and hyperparameter optimization techniques. The current challenges in relation to the application of DL in medicine and future research directions are outlined in Section 5. Finally, in Section 6 our conclusions are presented.
Section snippets
Deep learning models
In this section we will provide a comprehensive overview of DL models applied in medicine. Starting from an in-depth study of the literature, we will present the main families of DL architectures: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Autoencoders (AE), Generative Adversarial Networks (GAN), Deep Belief Networks (DBN) and Hybrid Architectures (HA) (see Fig. 3).
In the following subsections of this section, the various DL models will be discussed: Section 2.1 the
Applications fields of DL in medicine
DL achieves a remarkable accuracy and quality of its results thanks to its multi-layer architecture, which is able to obtain a high level of abstraction by working with large data samples. For this reason, it is gaining a great popularity in all fields where the process of information extraction from data involves various problems, such as the medical sector [93], [94], [95].
In detail, the analysis of medical data encounters three main issues, which are summarized in the following discussion.
Data structure and hyperparameters optimization
DL methodologies in medicine, as well as in all in the other fields where these techniques can be applied, often require the analysis of some “algorithmic” problems that can arise from data or from the algorithm itself. For these reasons, in the following Section a brief discussion on two of the main issues involved in the usage of Neural network approaches are discussed: on one side the format of stored data usually affect the class of algorithms that can be used on a specific problem, so in
Challenges and future research directions
DL is becoming the new paradigm in the analysis of medical data, as confirmed by the results discussed in Section 3. In addition, in recent years many other medical fields are beginning to benefit from the ability of these models to extract information from very different kinds of data. However, the complexity of DL models, the heterogeneity of medical data and the necessary interaction between machines and humans pose several issues, which must be taken into account in any assessment of future
Conclusions
DL is changing the cultural paradigm of medicine: its applications could become increasingly indispensable in terms of providing answers in contexts of high complexity and uncertainty and in order to allow doctors to have more time to take care of the medical needs of their patients. However, data are not values; any intervention based on data must be personalized, also taking into account the frequently contradictory nature of the knowledge provided in the literature. DL will be useful mainly
CRediT authorship contribution statement
Francesco Piccialli: Conceptualization, Methodology, Investigation, Writing, Visualization, Supervision. Vittorio Di Somma: Data curation, Writing, Investigation. Fabio Giampaolo: Writing, Resources, Review & editing. Salvatore Cuomo: Formal analysis, Writing - review & editing. Giancarlo Fortino: Writing - review & editing, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The authors dedicate this work to their friend and colleague Prof. Antonio Picariello, who passed away prematurely. The memory of his wonderful and kind soul will always remain in our hearts.
This work was supported by the CUP-in-un-click (CUP-in-One-Click) research, Italy project [Regione Campania - Bando RIS3 2018 - Fase 2]. The authors would like to thank the M.O.D.A.L. research laboratory (http://www.labdma.unina.it/index.php/modal/) for their efforts and support.
References (305)
- et al.
Bodycloud: A saas approach for community body sensor networks
Future Gener. Comput. Syst.
(2014) - et al.
Julia language in machine learning: Algorithms, applications, and open issues
Comp. Sci. Rev.
(2020) - et al.
A survey on deep learning in medical image analysis
Med. Image Anal.
(2017) - et al.
Survey on deep learning for radiotherapy
Comput. Biol. Med.
(2018) Deep learning in neural networks: An overview
Neural Netw.
(2015)- et al.
Deep learning for image-based cancer detection and diagnosis- a survey
Pattern Recognit.
(2018) - et al.
Deep transfer learning for modality classification of medical images
Information
(2017) - et al.
Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital x-ray mammograms
Comput. Methods Programs Biomed.
(2020) - et al.
Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning methodology
Comput. Biol. Med.
(2018) - et al.
A deep learning approach for path prediction in a location-based IoT system
Pervasive Mob. Comput.
(2020)
Predicting healthcare trajectories from medical records: A deep learning approach
J. Biomed. Inform.
A learning algorithm for Boltzmann machines
Cogn. Sci.
A study of deep convolutional auto-encoders for anomaly detection in videos
Pattern Recognit. Lett.
Learning multiple layers of representation
Trends Cogn. Sci.
Human emotion recognition using deep belief network architecture
Inf. Fusion
Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis
Med. Image Anal.
Deep learning-based segmentation of breast masses in dedicated breast CT imaging: Radiomic feature stability between radiologists and artificial intelligence
Comput. Biol. Med.
Deep learning and its applications to machine health monitoring
Mech. Syst. Signal Process.
Artificial intelligence in medicine: where are we now?
Acad. Radiol.
Deep Learning
Deep learning for heterogeneous medical data analysis
World Wide Web
A guide to deep learning in healthcare
Nat. Med.
Deep learning and alternative learning strategies for retrospective real-world clinical data
NPJ Digit. Med.
Deep learning: current and emerging applications in medicine and technology
IEEE J. Biomed. Health Inf.
Deep learning in omics: a survey and guideline
Brief. Funct. Genom.
Neural Networks and Deep Learning, Vol. 2018
Handwritten digit recognition with a back-propagation network
When deep learning meets metric learning: Remote sensing image scene classification via learning discriminative CNNs
IEEE Trans. Geosci. Remote Sens.
Object detection with deep learning: A review
IEEE Trans. Neural Netw. Learn. Syst.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
IEEE Trans. Pattern Anal. Mach. Intell.
Deep convolutional neural networks for image classification: A comprehensive review
Neural Comput.
Exploring deep learning models for overhead view multiple object detection
IEEE Internet Things J.
Large-scale video classification with convolutional neural networks
Imagenet classification with deep convolutional neural networks
U-net: Convolutional networks for biomedical image segmentation
Interpretation of intelligence in CNN-pooling processes: a methodological survey
Neural Comput. Appl.
Very deep convolutional networks for large-scale image recognition
Liver fibrosis classification based on transfer learning and FCNet for ultrasound images
Ieee Access
Scene text detection and segmentation based on cascaded convolution neural networks
IEEE Trans. Image Process.
CNN architectures for large-scale audio classification
Automatic brain tumor detection and segmentation using u-net based fully convolutional networks
3D U-Net: learning dense volumetric segmentation from sparse annotation
Unet++: A nested u-net architecture for medical image segmentation
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
IEEE Trans. Pattern Anal. Mach. Intell.
Rich feature hierarchies for accurate object detection and semantic segmentation
R-fcn: Object detection via region-based fully convolutional networks
Cited by (226)
Multiscale triplet spatial information fusion-based deep learning method to detect retinal pigment signs with fundus images
2024, Engineering Applications of Artificial IntelligenceContour-induced parallel graph reasoning for liver tumor segmentation
2024, Biomedical Signal Processing and ControlFusion of standard and ordinal dropout techniques to regularise deep models
2024, Information FusionSparse Dynamic Volume TransUNet with multi-level edge fusion for brain tumor segmentation
2024, Computers in Biology and MedicineFaceNet recognition algorithm subject to multiple constraints: Assessment of the performance
2024, Scientific African