A Trusted Friend in the Middle of the Night: End-User Perspectives on Artificial Intelligence Informed Software Systems as a Decision-Making Aid for Patients and Clinicians Navigating Uncertainty in Kidney Transplant

Abstract

Deceased donor kidney transplantation is often the only treatment for patients in end-stage renal disease and other life-threatening conditions. Kidney transplant processes can be fraught with uncertainty for both clinicians making critical decisions about whether to transplant an organ and the kidney recipients. There is potential for AI-informed software-based systems to support the activities of information-giving, decision-making, and waiting. This study analyses qualitative interviews to explore the user perspectives including those of both clinicians and transplant patients regarding this kind of decision making aid. Fourteen kidney transplant recipients and ten clinicians were recruited in a U.K transplant centre clinic. Data was collected via face-to-face and video-recorded semi-structured interviews and was analysed using a modified grounded-theory approach. Two patient themes were generated: ‘The murky waters of AI’ and ‘AI-driven tools could help transplant patients.’ The clinician themes included: ‘Understanding AI and the general perception around this technology,’ and ‘AI can be a friend to call on.’ The results highlight the possibility of an AI software programme to explain complex ideas to patients, by providing visual and graphical representations of AI-powered, individualised survival calculations or organ wait list times. The design and implementation of such tools must centre around trust in AI technology for clinicians and patients. The balance of staying on the waiting list or accepting an organ involves many complex factors but using AI-informed technology would be welcomed by patients and clinicians.

Appendix

Supplementary Material 1: Patient Participant Interview Schedule

Explainable machine learning models for clinical decision support in kidney transplant offering.

[For Internal Use Only]

Demographic details/introductory questions

• 1. What is your age?

• 2. What is your ethnicity?

• 3. When did you receive your kidney transplant?

• 4. Why did you need a kidney transplant?

• 5. What medical conditions do you currently suffer from? (i.e. diabetes, COPD, etc.)

Decision making around the transplant process.

• 1. When you discussed the transplant offering system with your doctor, how much information were you given about the process? Did you feel that you received an option on whether to accept verse decline an offer?

• 2. How did the clinician give you the information about your offer? Did they simply tell you an offer was made or did you have information on the quality of the organ itself?

• 3. Did your doctor say you were a difficult to match patient? Or were their discussions around the criteria needed to match you with a suitable donor organ?

• 4. Did you have any discussions around waiting list times during your transplant journey? If so, were you given estimated times for how long you would have to wait? If not, would you have liked to have this information?

• 5. Looking back on your transplant experience, how much information about the transplant process would you have liked to have? (i.e. I wished I had received less information at the time, what I received at the time was sufficient, or I wish I had been given more detail)

General Artificial/Machine Learning themed Questions.

Note: researcher to ask the first question to the participant. If the participant does not have any knowledge in AI, the researcher will elaborate and give a basic explanation of AI to include the following:

Artificial Intelligence (AI) is a type of computer programme that mimics human thinking in the way it processes information. It is able to form complex pathways and ‘think’ like humans. The way that AI works is sometimes understood by computer scientists but there are other instances that AI works in ways we can not explicitly understand. Researchers are using this type of technology to help solve complicated problems in a variety of fields, including medicine.

Additional Note: depending on participant’s answers, only Questions 1 and 2 to be asked if no previous knowledge of AI.

• 1. Do you have any knowledge of Artificial Intelligence (AI)? And what level of knowledge do you have?

• 2. How much interest in AI do you have? If you are interested in AI, where did you first hear about it? (news programme, article online, etc.)

• 3. What is your awareness and views on AI in general?

• 4. What do you think are the positive aspects of AI?

• 5. Do you think there are any negative aspects of AI?

Difficult to ask questions about AI if patient is unaware of AI in general – may need to relate it to mathematical tool.

Research specific questions and Clinical Decision Support Tool.

• 1. What aspects of Artificial Intelligence (AI) do you have concerns around? What would stop you from accepting this technology to help make decisions by doctors? (i.e. around transplanted organs)

• 2. How would you feel about AI technology being used to assist clinicians make a decision about your transplants?

• 3. How transparent would you need the AI tool (in calculating your organ transplant or waitlist times) to be for you to feel comfortable? (i.e. I would need to know exactly how the AI model calculated the survival, I would only need to know some of the way the model calculated this information, I would not need to know how the tool worked)

• 4. How would you feel about AI technology helping doctors make predictions on how long your waiting time before an organ may be offered to you?

• 5. If AI could be used to predict your organ survival as well as waitlist times if you decided not to accept an organ, how would you like to have this information presented to you? (i.e. visual representation, graphs, numeric data)?

• 6. Is there anything you would like to discuss that we haven’t discussed today?

Supplementary Material 2: Clinician Interview Schedule

Explainable machine learning models for clinical decision support in kidney transplant offering.

[For internal use only]

Demographic details/introductory questions

• 1. What is your current role?

• 2. How many years have you been involved in kidney transplants?

• 3. How long have you worked at the Oxford Transplant Centre?

Decision making around the transplant process

• 1. When discussing the transplant offering system with a kidney transplant patient, how much information do you tell the patient about the process? Do you ask them how much information they would like to receive?

• 2. How do you present the information about a potential kidney offer? Do you look through the clinical data first and make a decision about whether this is an appropriate offer or do you involve the patient in the decision-making process?

• 3. Do you let patients know if they may be a difficult to match patient?

• 4. Do you discuss the pros and cons of accepting a potential donor organ verse continue to stay on the wait list?

• 5. Do you tell patients how long they may have to wait on the waiting list? (even if this is an estimation)

• 6. If a marginal kidney is offered, do you let the patient know this information? Do you involve them in the decision-making process to accept/reject such an organ?

General Artificial/Machine Learning themed Questions

Note: researcher to ask the first question to the participant. If the participant does not have any knowledge in AI, the researcher will elaborate and give a basic explanation of AI to include the following:

Artificial Intelligence (AI) is a type of computer programme that aims at providing computer programmes that mimic human thinking in the way they processes information. The way that AI works is sometimes understood by computer scientists but there are other instances that AI works in ways we can not explicitly understand. Researchers are using this type of technology to help solve complicated problems in a variety of fields, including medicine.

One example of AI is in risk prediction scoring. Similar to regression models there are input variables that can be used to determine outcomes. However, in AI we are not always aware of the weighting of each variable in the algorithm.

Additional Note: depending on participant’s answers, only Questions 1 and 2 to be asked if no previous knowledge of AI.

• 1. Do you have any knowledge of Artificial Intelligence (AI)? And what level of knowledge do you have?

• 2. How much interest in AI do you have? If you are interested in AI, where did you first hear about it? (news programme, article online, etc.)

• 3. What is your awareness and views on AI in general?

• 4. What do you think are the positive aspects of AI?

• 5. Do you think there are any negative aspects of AI?

• 6. Are you aware of AI being used in clinical practice? If so, what areas?

• 7. Have you heard of any clinicians or research groups using AI in the transplant decision making process? If so, what did you think about this?

Research specific questions and Clinical Decision Support Tool

• 1. Do you use any calculators or risk assessment tools in your clinical practice? What do you like/dislike about them?

• 2. What aspects of Artificial Intelligence (AI) do you have concerns around? What would stop you from accepting this technology to help you make decisions? (i.e. around transplanted organs)

• 3. How would you feel about AI technology being used to assist you to make a decision about your patients’ transplants?

• 4. How transparent would you need the AI tool (in calculating the organ transplant survival or waitlist times) to be for you to feel comfortable? (i.e. I would need to know exactly how the AI model calculated the survival, I would only need to know some of the way the model calculated this information, I would not need to know how the tool worked)

• 5. If AI could be used to predict patient organ survival as well as waitlist times if you decided to reject an organ in the form of a clinical decision support tool, how would you like to have this information presented to you? (i.e. visual representation, graphs, numeric data)?

• 6. How much detail do you think your patients would want from the tool? (i.e. very detailed, somewhat detailed, minimal detail)

• 7. In what format should the clinical decision support tool take? (i.e. web-based, app-based)

• 8. Would a tool be useful in the informed consent process? (similar to P-POSSOM score for laparotomy)

• 9. Is there anything you would like to discuss that we haven’t discussed today?