Designing medical artificial intelligence for in- and out-groups
Section snippets
Medical AI design
The design of AI is contingent on the characteristic of its specific task (Jordan & Mitchell, 2015). The more complex the task is, the more sophisticated it should be—feeding more data, training with more complex algorithms, and so on. For instance, in image recognition, it is easy to construct an AI system to differentiate images of animals from humans. Nevertheless, to distinguish a white wolf and a breed of wolf-like white dog accurately, the AI system should be trained with deep-learning
The present research
Employing an experimental approach is vital in identifying how specific factors shape people's decision-making, like designing a product (Herd & Mehta, 2019) or providing healthcare (Godager & Wiesen, 2013; Schram & Sonnemans, 2011). For example, to identify physicians' altruism toward patients' health benefits, in experimental research conducted by Godager and Wiesen (2013), medical students were asked to adopt the role of physicians and decide the quantity of medical services for a given
Study 1
Study 1 tested how the target group (in-vs. out-groups) affects people's decision to adopt human doctors' assistance when designing medical AI products. Previous research has shown that combining human training in the AI system is an effective way to increase AI's performance (Holzinger, 2016). Studies targeting potential medical AI users also highlighted that robots should be combined with humans, instead of replacing humans (Lehoux & Grimard, 2018). Consumers demonstrated resistance to
Study 2
In Study 2, the specific out-group was changed from American patients to Chinese patients to increase the generalization of this effect. The authors predicted that Americans would perceive out-groups (Chinese patients) as homogeneous. The ignorance of out-group diversity would lead to a lower likelihood of adopting human doctors’ assistance in the medical AI system for out-groups than for in-groups.
Study 3
The goals of Study 3 were several-fold. First, the authors investigated a new form of dealing with the diversity of patients’ medical needs: the size of database. Participants were asked to decide the size of the database. The larger the database, the more diverse the medical samples would be, and, in turn, the better the medical AI service would be at dealing with rare medical cases. The authors sought to document inequity between out- and in-groups.
Second, whereas Studies 1 and 2 involved the
General discussion
Sophisticated AI systems have been designed to deliver worldwide access to healthcare. However, before implementing them in the medical situation, profound questions should be raised. An error system, for example, that makes a minor mistake, could be fatal and disastrous (Synced, 2018). This research sought to explore various designs of medical AI between in- and out-groups. Studies 1 (pre-registered experiment in China; N = 191) and 2 (experiment in the U.S.; N = 190) revealed that people were
Conclusion
Medical AI is gaining tremendous popularity in promoting worldwide healthcare. Current literature lacks a systematic consideration of how individuals perceive out-groups when designing medical AI. This work documented an inequity between in- and out-groups when designing medical AI. People tend to demonstrate an out-group homogeneity effect, perceiving out-groups as similar. This simplified process leads to people's underestimation of the diversity of out-group members' medical needs and
Author contribution
W.L. and X.Z. designed the studies; W.L., X.Z. and Q.Y. conducted the studies and analyzed the data; W.L., X.Z. and Q.Y. discussed conceptual issues and wrote the manuscript.
Funding
This work was supported by the Key Program and General Program of the National Natural Science Foundation of China, China [grant numbers 31871095; 71672169; 71925005; 71974170]; Soft Science Project of Science Technology Department of Zhejiang Province of China [grant number 2019C35025]; Youth Project of the Center for Health Policy and Hospital Management of Zhejiang University, China [grant number 2019WSZ007]; Leading Innovative and Entrepreneur Team Introduction Program of Zhejiang, China
Declaration of competing interest
None.
References (63)
Intergroup perception in the social context: The effects of social status and group membership on perceived out-group homogeneity and ethnocentrism
Journal of Experimental Social Psychology
(2001)- et al.
Skin color as a predictor of mental health in young Latinx children
Social Science & Medicine
(2019) Machine learning approaches in medical image analysis: From detection to diagnosis
Medical Image Analysis
(2016)- et al.
An interactive web-based dashboard to track COVID-19 in real time
The Lancet Infectious Diseases
(2020) - et al.
Reducing racial disparities in pain treatment: The role of empathy and perspective-taking
Pain
(2011) - et al.
Evaluations of an artificial intelligence instructor's voice: Social Identity Theory in human-robot interactions
Computers in Human Behavior
(2019) - et al.
Profit or patients' health benefit? Exploring the heterogeneity in physician altruism
Journal of Health Economics
(2013) Cybersecurity, social engineering, artificial intelligence, technological addictions: Societal challenges for the coming decade
Computers in Human Behavior
(2020)- et al.
Cancer risk in socially marginalised women: An exploratory study
Social Science & Medicine
(2019) - et al.
Other-regarding behavior and motivation in health care provision: An experiment with medical and non-medical students
Social Science & Medicine
(2014)