Machine learning for medical diagnosis: history, state of the art and perspective

doi:10.1016/S0933-3657(01)00077-X

Artificial Intelligence in Medicine

Volume 23, Issue 1, August 2001, Pages 89-109

https://doi.org/10.1016/S0933-3657(01)00077-X Get rights and content

Abstract

The paper provides an overview of the development of intelligent data analysis in medicine from a machine learning perspective: a historical view, a state-of-the-art view, and a view on some future trends in this subfield of applied artificial intelligence. The paper is not intended to provide a comprehensive overview but rather describes some subareas and directions which from my personal point of view seem to be important for applying machine learning in medical diagnosis. In the historical overview, I emphasize the naive Bayesian classifier, neural networks and decision trees. I present a comparison of some state-of-the-art systems, representatives from each branch of machine learning, when applied to several medical diagnostic tasks. The future trends are illustrated by two case studies. The first describes a recently developed method for dealing with reliability of decisions of classifiers, which seems to be promising for intelligent data analysis in medicine. The second describes an approach to using machine learning in order to verify some unexplained phenomena from complementary medicine, which is not (yet) approved by the orthodox medical community but could in the future play an important role in overall medical diagnosis and treatment.

Introduction

Artificial intelligence is a part of computer science that tries to make computers more intelligent. One of the basic requirements for any intelligent behavior is learning. Most of the researchers today agree that there is no intelligence without learning. Therefore, machine learning [1], [2], [3], [4] is one of major branches of artificial intelligence and, indeed, it is one of the most rapidly developing subfields of AI research.

Machine learning algorithms were from the very beginning designed and used to analyze medical datasets. Today, machine learning provides several indispensible tools for intelligent data analysis. Especially in the last few years, the digital revolution provided relatively inexpensive and available means to collect and store the data. Modern hospitals are well equipped with monitoring and other data collection devices, and data is gathered and shared in large information systems. Machine learning technology is currently well suited for analyzing medical data, and in particular there is a lot of work done in medical diagnosis in small specialized diagnostic problems.

Data about correct diagnoses are often available in the form of medical records in specialized hospitals or their departments. All that has to be done is to input the patient records with known correct diagnosis into a computer program to run a learning algorithm. This is of course an oversimplification, but in principle, the medical diagnostic knowledge can be automatically derived from the description of cases solved in the past. The derived classifier can then be used either to assist the physician when diagnosing new patients in order to improve the diagnostic speed, accuracy and/or reliability, or to train students or physicians non-specialists to diagnose patients in a special diagnostic problem.

The aim of this paper is to provide an overview of the development of the intelligent data analysis in medicine from a machine learning perspective: a historical view, a state-of-the-art view and a view on some future trends in this subfield of applied artificial intelligence, which are, respectively, described in 2 Historical overview, 3 State of the art, 4 Future trends — two case studies. None of the three sections is intended to provide a comprehensive overview, but rather describe some subeareas and directions which from my personal point of view seem to be important for medical diagnosis. In the historical overview, I emphasize the naive Bayesian classifier, neural networks, and decision trees. Section 3 presents a comparison of some state-of-the-art systems, one or two representatives from each branch of machine learning, when applied to several medical diagnostic tasks. The future trends are illustrated by two case studies. Section 4.1 describes a recently developed method for dealing with reliability of decisions of classifiers, which seems to be promising for intelligent data analysis in medicine. Section 4.2 describes an approach to using machine learning in order to verify some unexplained phenomena from complementary medicine, which is not (yet) approved by the orthodox medical community, but could in the future play an important role in overall medical diagnosis and treatment.

Section snippets

Historical overview

As soon as electronic computers came into use in the 1950s and 1960s, the algorithms were developed that enabled modeling and analyzing large sets of data. From the very beginning, three major branches of machine learning emerged. Classical work in symbolic learning is described by Hunt et al. [5], in statistical methods by Nilsson [6], and in neural networks by Rosenblatt [7]. Through the years, all three branches developed advanced methods [2]: statistical or pattern recognition methods, such

State of the art

In this section, we give a description of specific requirements that any machine learning system has to satisfy in order to be used in the development of applications in medical diagnosis. Several learning algorithms are briefly described. We compared the performance of all the algorithms on several medical diagnostic and prognostic problems, and their appropriateness for applications in medical diagnosis is discussed.

Future trends — two case studies

There are many directions in which future development of machine learning in medical diagnosis may take place. Some may rely on new trends in computer technology or technology of medical equipment, however, probably more important is going to be the development of new machine learning algorithms and the philosophy of medical diagnosis. We do not want to speculate all possible trends. Instead, we describe two case studies that illustrate the new trends in the development of machine learning

Discussion

The historical development of machine learning and its applications in medical diagnosis shows that from simple and straightforward to use algorithms, systems and methodology have emerged that enable advanced and sophisticated data analysis. In the future, intelligent data analysis will play even a more important role due to the huge amount of information produced and stored by modern technology. Current machine learning algorithms provide tools that can significantly help medical practitioners

Acknowledgements

Special thanks to Ivan Bratko, Matjaž Kukar, and Nada Lavrač for longterm joint work on projects related to intelligent data analysis in medicine. Experiments with the Kirlian camera were done with the invaluable help and support from Matjaž Bevk, Zoran Bosnić, Tom Chalko, Minnie Hein, my wife Irena, Milan Mladženović, Barbara Novak, Petar Papuga, Vili Poznik, Bor Prihavec, Marko Robnik-Šikonja, Aleksander Sadikov, Danijel Skočaj, Slobodan Stanojević, Tatjana Zrimec, and many others. I thank

References (81)

M. Kukar et al.
Machine learning in prognostics of the femoral neck fracture recovery
Artif. Intell. Med.
(1996)
M. Kukar et al.
Analysing and improving the diagnosis of ishaemic heart disease with machine learning
Artif. Intell. Med.
(1999)
Shavlik JW, Dietterich TG, editors. Readings in machine learning. Los Altos, CA: Morgan Kaufmann,...
Michie D, Spiegelhalter DJ, Taylor CC, editors. Machine learning, neural and statistical classification. Chichester,...
Mitchell T. Machine learning. New York: McGraw-Hill,...
Michalski RS, Bratko I, Kubat M, editors. Machine learning, data mining and knowledge discovery: methods and...
Hunt E, Martin J, Stone P. Experiments in induction. New York: Academic Press,...
Nilsson N. Learning machines. New York: McGraw-Hill,...
Rosenblatt F. Principles of neurodynamics. Washington, DC: Spartan Books,...
Kononenko I, Bratko I, Kukar M. Application of machine learning to medical diagnosis. In: Michalski RS, Bratko I, Kubat...

Lavrač N, Džeroski S. Inductive logic programming. Chichester, UK: Ellis Horwood,...

Pompe U, Kononenko I. Probabilistic first-order classification. In: Lavrač N, Džeroski S, editors. Proceedings of the...

D.J. Spiegelhalter et al.

Bayesian analysis in expert systems

Stat. Sci.

(1993)

Good IJ. Probability and the weighing of evidence. London: Charles Griffin,...

Good IJ. The estimation of probabilities — an essay on modern Bayesian methods. Cambridge: MIT Press,...

Kononenko I, Bratko I, Roškar E. Experiments in automatic learning of medical diagnostic rules. In: Proceedings of the...

Cestnik B, Kononenko I, Bratko I. ASSISTANT 86: a knowledge elicitation tool for sophisticated users. In: Bratko I,...

Kononenko I. Interpretation of neural networks decisions. In: Proceedings of the IASTED International Conference on...

I. Kononenko

Inductive and Bayesian learning in medical diagnosis

Appl. Artif. Intell.

(1993)

Cestnik B. Estimating probabilities: a crucial task in machine learning. In: Proceedings of the European Conference on...

Kononenko I. Semi-naive Bayesian classifier. In: Kodratoff Y, editor. Proceedings of the European Working Session on...

I. Kononenko

Naive Bayesian classifier and continuous attributes

Informatica

(1992)

Langley P. Induction of recursive Bayesian classifiers. In: Proceedings of the European Conference on Machine Learning,...

Pazzani M. Searching for dependencies in Bayesian classifiers. Artificial intelligence and statistics IV. Lecture notes...

Kohavi R, Becker B, Sommerfield D. Making sense of simple Bayes. Technical report, Data Mining and Visualization Group,...

Minsky M. In: Papert S, editor. Perceptrons. Cambridge, MA: MIT Press,...

J.J. Hopfield

Neural networks and physical systems with emergent collective computational abilities

Proc. Natl. Acad. Sci.

(1982)

J.J. Hopfield

Neurons with graded response have collective computational properties like those of two-state neurons

Proc. Natl. Acad. Sci.

(1984)

Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: Rumelhart DE,...

Haykin S. Neural networks: a comprehensive foundation. New York: Macmillan,...

I. Kononenko

Bayesian neural networks

Biol. Cybernetics J.

(1989)

Quinlan JR. Discovering rules from large collections of examples. In: Michie D, editor. Expert systems in the...

R.S. Michalski et al.

Learning by being told and learning from examples: an experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis

Int. J. Policy Anal. Inform. Syst.

(1980)

I. Bratko et al.

An experiment in automatic learning of diagnostic rules

Informatica (Ljubljana)

(1980)

Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and regression trees. Belmont, CA: Wadsworth,...

Elomaa T, Holsti N. An experimental comparison of inducing decision trees and decision lists in noisy domains. In:...

Lesmo L, Saitta L, Torasso P. Learning of fuzzy production rules for medical diagnoses. In: Gupta MM, Sanchez E,...

K.A. Horn et al.

An expert system for the interpretation of thyroid assays in a clinical laboratory

Aust. Comput. J.

(1985)

Hojker S, Kononenko I, Jauk A, Fidler V, Porenta M. Expert system’s development in the management of thyroid diseases....

Quinlan R, Compton P, Horn KA, Lazarus L. Inductive knowledge acquisition: a case study. In: Quinlan JR, editor....

Cited by (1219)

Paradigm shift from Artificial Neural Networks (ANNs) to deep Convolutional Neural Networks (DCNNs) in the field of medical image processing
2024, Expert Systems with Applications
Images and other types of unstructural data in the medical domain are rapidly becoming data-intensive. Actionable insights from these complex data present new opportunities but also pose new challenges for classification or segmentation of unstructural data sources. Over the years, medical problems have been solved by combining traditional statistical methods with image processing methods. Both the increase in the size of the data and the increase in the resolution are among the factors that shape the ongoing improvements in artificial intelligence (AI), particularly concerning deep learning (DL) techniques for evaluation of these medical data to identify, classify, and quantify patterns for clinical needs. At this point, it is important to understand how Artificial Neural Networks (ANNs), which are an important milestone in interpreting big data, transform into Deep Convolutional Neural Networks (DCNNs) and to predict where the change will go. We aimed to explain the needs of these stages in medical image processing through the studies in the literature. At the same time, information is provided about the studies that lead to paradigm shift and try to solve the image related medical problems by using DCNNs. With the increase in the knowledge of medical doctors on this subject, it will be possible to look at the solution of new problems in computer science from different perspectives.
Artificial intelligence in mobile forensics: A survey of current status, a use case analysis and AI alignment objectives
2024, Forensic Science International: Digital Investigation
As the capabilities and utility of Artificial Intelligence and Machine Learning systems continue to improve, they are expected to have an increasingly powerful influence in the digital forensic investigation process. The concurrent proliferation of mobile devices and rapid increase of forensic value of related artifacts creates the requirement for a comprehensive review of the current status of artificial intelligence software usage and usefulness in Mobile Forensics. In this context, we conducted a survey to evaluate the characteristics and properties of AI functions in mobile forensic software from the practitioners' perspective and enhance understanding to the work in the field. In this study, we evaluated the performance of image categorization software in digital forensics using a variety of evaluation metrics including accuracy, precision, recall, and F1-score, as well as the confusion matrix. In this research we also identify and integrate theoretical principles to conceptualize an AI Alignment framework pertaining to Mobile Forensics and Digital Forensics in general, in order to accurately determine specific AI strategy objectives and potential solutions to the current technical and administrative landscape. We emphasized the importance of interpretability and transparency in AI systems and the need for a comprehensive approach to understanding the reasoning behind the software's decisions. Additionally, we highlighted the importance of robustness in image categorization software, as well as the consideration of AI governance and standardized procedures concepts. Our results show that the accuracy and robustness of the image categorization software have a significant impact on the outcome of legal cases and that the software should be designed with interpretability, transparency, and robustness in mind. Through the examination of the survey responses, the evaluation of the image categorization software and research literature, we explore existing and potential approaches to aligned Artificial Intelligence and analyze their contribution to the forensic examination of cases.
RNFLT2Vec: Artifact-corrected representation learning for retinal nerve fiber layer thickness maps
2024, Medical Image Analysis
Optical coherence tomography imaging provides a crucial clinical measurement for diagnosing and monitoring glaucoma through the two-dimensional retinal nerve fiber layer (RNFL) thickness (RNFLT) map. Researchers have been increasingly using neural models to extract meaningful features from the RNFLT map, aiming to identify biomarkers for glaucoma and its progression. However, accurately representing the RNFLT map features relevant to glaucoma is challenging due to significant variations in retinal anatomy among individuals, which confound the pathological thinning of the RNFL. Moreover, the presence of artifacts in the RNFLT map, caused by segmentation errors in the context of degraded image quality and defective imaging procedures, further complicates the task. In this paper, we propose a general framework called RNFLT2Vec for unsupervised learning of vectorized feature representations from RNFLT maps. Our method includes an artifact correction component that learns to rectify RNFLT values at artifact locations, producing a representation reflecting the RNFLT map without artifacts. Additionally, we incorporate two regularization techniques to encourage discriminative representation learning. Firstly, we introduce a contrastive learning-based regularization to capture the similarities and dissimilarities between RNFLT maps. Secondly, we employ a consistency learning-based regularization to align pairwise distances of RNFLT maps with their corresponding thickness distributions. Through extensive experiments on a large-scale real-world dataset, we demonstrate the superiority of RNFLT2Vec in three different clinical tasks: RNFLT pattern discovery, glaucoma detection, and visual field prediction. Our results validate the effectiveness of our framework and its potential to contribute to a better understanding and diagnosis of glaucoma.
Parkinson's disease diagnosis using deep learning: A bibliometric analysis and literature review
2024, Ageing Research Reviews
Parkinson’s Disease (PD) is a progressive neurodegenerative illness triggered by decreased dopamine secretion. Deep Learning (DL) has gained substantial attention in PD diagnosis research, with an increase in the number of published papers in this discipline. PD detection using DL has presented more promising outcomes as compared with common machine learning approaches. This article aims to conduct a bibliometric analysis and a literature review focusing on the prominent developments taking place in this area. To achieve the target of the study, we retrieved and analyzed the available research papers in the Scopus database. Following that, we conducted a bibliometric analysis to inspect the structure of keywords, authors, and countries in the surveyed studies by providing visual representations of the bibliometric data using VOSviewer software. The study also provides an in-depth review of the literature focusing on different indicators of PD, deployed approaches, and performance metrics. The outcomes indicate the firm development of PD diagnosis using DL approaches over time and a large diversity of studies worldwide. Additionally, the literature review presented a research gap in DL approaches related to incremental learning, particularly in relation to big data analysis.
Combining theoretical modelling and machine learning approaches: The case of teamwork effects on individual effort expenditure
2024, New Ideas in Psychology
Machine learning modelling of psychological processes is often considered as competing alternative to theoretical modelling. In contrast, the current study explores potential synergetic effects of these two general approaches both for predictive accuracy and theoretical understanding. Theoretical models have high explanatory value but can have weak predictive power. Machine learning models have high predictive power but low transparency and require large amounts of data and computational power. The combination of machine learning and theoretical models may yield both higher predictive accuracy as well as higher explanatory value and lower requirements of data and computational power as compared to either of the two approaches alone. We examine our assumptions in the field of team motivation, using archival performance data from 1,425,926 individual and relay races of swimming competitions. While the results revealed better prediction of the machine learning model, an exploration of the machine learning model with explainable artificial intelligence methods offered new insights also for the theoretical model. Finally, the combination of machine learning and theoretical modelling required less computational power than the machine learning approach alone, but not less data for building the model.
Novel three-way decision model in medical diagnosis based on inexact reasoning
2024, Engineering Applications of Artificial Intelligence
According to the clinical symptoms, a patient exhibits and medical domain knowledge, determining the type of disease the patient has is essential for medical diagnosis. Uncertainty is the nature of medical diagnosis. The intuitionistic fuzzy set (IFS) is one of the effective tools to deal with uncertainty problems. In recent years, IFS has been widely used to deal with medical diagnosis problems. The existing mainstream approach is to rank the underlying diseases and then assign the first-ranked disease to the patient. These methods cannot deal with situations where patients have multiple diseases at the same time or no disease at all. In addition, these methods may generate unreasonable diagnostic results in some cases. Consider clinical symptoms as evidence that a patient has or does not have the disease. With the increase in clinical symptoms, physicians will increase their belief or disbelief that the patient has the disease. Therefore, based on the method of inexact reasoning, this paper proposed an inexact reasoning model in medical diagnosis, namely IRM-MD, which can rank the underlying diseases and avoid generating unreasonable diagnostic results. Furthermore, a three-way decision model for medical diagnosis based on IRM-MD, namely 3WDM-IRM, is proposed. This model can effectively deal with cases where patients have multiple conditions at the same time or none at all, and prevent missed diagnoses and misdiagnoses. Finally, the numerical experiment results show that (a)the results generated by the IRM-MD model are similar to those of other existing models, and (b)the 3WDM-IRM model can effectively identify whether a patient has a disease or not, and identify multiple diseases that the patient has at the same time.

View all citing articles on Scopus

View full text

Machine learning for medical diagnosis: history, state of the art and perspective

Abstract

Introduction

Section snippets

Historical overview

State of the art

Future trends — two case studies

Discussion

Acknowledgements

Artif. Intell. Med.

Artif. Intell. Med.

Bayesian analysis in expert systems

Stat. Sci.

Inductive and Bayesian learning in medical diagnosis

Appl. Artif. Intell.

Naive Bayesian classifier and continuous attributes

Informatica

Neural networks and physical systems with emergent collective computational abilities

Proc. Natl. Acad. Sci.

Neurons with graded response have collective computational properties like those of two-state neurons

Proc. Natl. Acad. Sci.

Bayesian neural networks

Biol. Cybernetics J.

Learning by being told and learning from examples: an experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis

Int. J. Policy Anal. Inform. Syst.

An experiment in automatic learning of diagnostic rules

Informatica (Ljubljana)

An expert system for the interpretation of thyroid assays in a clinical laboratory

Aust. Comput. J.