Collective intelligence in medical diagnosis systems: A case study

Abstract

Diagnosing a patient's condition is one of the most important and challenging tasks in medicine. We present a study of the application of collective intelligence in medical diagnosis by applying consensus methods. We compared the accuracy obtained with this method against the diagnostics accuracy reached through the knowledge of a single expert. We used the ontological structures of ten diseases. Two knowledge bases were created by placing five diseases into each knowledge base. We conducted two experiments, one with an empty knowledge base and the other with a populated knowledge base. For both experiments, five experts added and/or eliminated signs/symptoms and diagnostic tests for each disease. After this process, the individual knowledge bases were built based on the output of the consensus methods. In order to perform the evaluation, we compared the number of items for each disease in the agreed knowledge bases against the number of items in the GS (Gold Standard). We identified that, while the number of items in each knowledge base is higher, the consensus level is lower. In all cases, the lowest level of agreement (20%) exceeded the number of signs that are in the GS. In addition, when all experts agreed, the number of items decreased. The use of collective intelligence can be used to increase the consensus of physicians. This is because, by using consensus, physicians can gather more information and knowledge than when obtaining information and knowledge from knowledge bases fed or populated from the knowledge found in the literature, and, at the same time, they can keep updated and collaborate dynamically.

Introduction

In medical practice, first-contact medical care is the main entrance to health services, providing continuous and comprehensive personal attention to patients. Therefore, a proper communication channel between the first and second level of medical care is needed. Physicians use interconsulting as the main element to build this communication channel [1], [2], [3]. Mainly, this practice allows them to share information to reach a consensus about a diagnosis or treatment [4]. The ability of a physician to diagnose a patient's condition depends on several factors, such as knowledge, training, experience, available resources, communication skills and, often, instinct [5], [6], [7], [8], [9]. Sharing information is the act of disseminating valuable knowledge gained with other members within an organisation. This activity of physicians in hospitals can potentially generate huge profits and is essential to succeed and survive in competitive environments [10].

This research is based on two concepts closely related to medicine and health care: consensus methods and collective intelligence. The use of consensus methods to solve problems related to health and medicine has been increasing. These methods define levels of agreement between individuals, and, if they are used properly, it is possible to create structured environments that provide the best information that allows experts to solve problems in a more conclusive manner [11]. In [12], Heylighen defines collective intelligence as the ability of a group to solve more problems than its individual members can separately. The basic idea is that a group of individuals can be smarter together than separated. In the same manner, Lévy explains that the term Collaborative Intelligence encompasses Collective Intelligence, defined as a form of universally distributed intelligence, constantly enhanced, coordinated in real time and employing effective mobilisation skills [16].

Collaborative intelligence comes from the synergy created when individuals interact with each other following simple rules. However, collective intelligence also has some limitations. One of the main problems is that an expert may incorrectly interpret the ideas of another expert due to differences in experience, knowledge or terms used. In medicine, this problem is known as overlapping [12], [13], [14], [15].

We present a case study where physicians employed consensus methods and collective intelligence in order to obtain a medical diagnosis. We compared the diagnosis accuracy that was reached by applying collective intelligence against the diagnosis accuracy reached through the knowledge of a single expert. The experiment was supported by a Diagnosis Decision Support System (DDSS). In addition, based on the information collected, we analysed the way in which medical diagnosis was influenced by the physicians’ different opinions. All this work helped us to test the hypothesis that different levels of intelligence between a group of people and its individual members exist. Additionally, we found that these intelligence levels can be measured and used to predict the performance of groups in a variety of tasks.

The rest of this paper is structured as follows. Section 2 presents the state of the art focusing on consensus methods, which are traditionally used in the medical field. In Section 3, we outline the research description and methods used. In Section 4, we show the main results obtained. Finally, in Section 5, we present our conclusions about the results and suggest future research directions and challenges.