Semantic knowledge modeling and evaluation of argument Theory to develop dialogue based patient education systems for chronic disease Self-Management

https://doi.org/10.1016/j.ijmedinf.2022.104693Get rights and content

Highlights

  • Use of AI-based argument theory to structure patient education dialogues.

  • Developed ontology-based knowledge model to represent dialogue structure and content.

  • Education content is structured as arguments that offers evidence for recommendations.

  • Developed dialogue-based patient education system for Juvenile Idiopathic Arthritis.

  • Qualitative evaluation showed the utility of the dialogue system for patient education.

Abstract

Background

To improve understanding as well as uptake of health educational material, it should be tailored to informational needs, offer intuitive modes of interaction, and present credible evidence for health claims. Dialogue systems go some way in meeting these requirements, as they emulate interactive and intuitive person-to-person communication. However, most works do not offer a formal model nor modelling process to structure dialogue content, and do not focus on ensuring credibility.

Methods

We propose an Extended Model of Argument (EMA) dialogue model and modelling process to support educational dialogue systems. In this dialogue model, computerized arguments directly offer evidence for health claims. EMA further offers “dialogue by design”, where argument structures and interrelations are dynamically leveraged to offer dialogues, instead of relying on predefining discourse flows. We implemented an EMA-based dialogue education system for Juvenile Idiopathic Arthritis (JIA). We performed a qualitative evaluation with JIA health experts involving a Cognitive Walkthrough and Semi-Structured Interview. We applied Directed Content Analysis using categories from the O’Grady framework, and coded sub-themes within those categories using Grounded Theory.

Results

We identified 6 sub-themes within the participant feedback pertaining to Quality, Credibility, and Utility. Participants attached strong importance to credibility and found the dialogue system to be a flexible educational tool. Some participants suggested sorting educational items by importance, and presenting only salient knowledge associations to reduce dialogue complexity.

Conclusion

Overall, our qualitative evaluation confirmed the following: the ability of EMA to offer credible and appropriate dialogues; and, in general, the utility of dialogue systems to educate JIA patients and their families. In future work, we will revise the system based on evaluation feedback, and perform a more extensive evaluation with patients and caregivers.

Introduction

The successful long-term treatment of chronic conditions relies on the empowerment of patients and caregivers to (self-)manage non-acute symptoms. This requires comprehensive knowledge on the illness, hence, chronic disease patients are typically provided with educational resources such as websites and pamphlets [1]. Nevertheless, the static and non-interactive nature of these resources reduces their effectiveness for education. Firstly, patients have different informational needs depending on the situation (lookup vs. browsing), requiring appropriate types of interactivity. Moreover, healthcare users have expressed a strong interest in credibility of educational material [1] –this has become paramount with the current increase in online health-related material from non-experts.

Dialogue systems (a.k.a. conversational systems or chatbots) are intelligent computer systems that emulate an interactive, intuitive person-to-person communication [2]. Dialogue systems can make health educational content more dynamic, personal and interactive, thus improving the material’s understanding and uptake [3]. Multiple dialogue systems have been developed to support self-management for various conditions [3], including cancer [4], [5], [6], mental health [7], [8], [9], [10], [11], hypertension [12], and diabetes [13]. Based on our review of these papers, we found that current dialogue systems do not seem to structure the dialogue based on a formal dialogue model to offer personalized, interactive dialogues for health education. A similar observation was made by Tudor Car et al [3]. Instead, the flow of the dialogue is typically orchestrated by a separate decision tree on top of the educational material: the conversation moves through a linear path, with users choosing pre-determined options at each branch, with no focus on emphasizing credibility of the material.

In this paper, we introduce the use of Argument Theory as a formal dialogue model to structure the contents of a credible and interactive patient education dialogue, based on evidence-driven educational content. A novel aspect of our approach is dialogue by design, where credible dialogues are formulated directly from the structured arguments: we dynamically leverage internal argument structures and their relations to formulate dialogues. This removes the need for predefined linear discourses, as is the case in most health-related dialogue systems. Dialogue by design further offers multiple types of interactivity, where users can seek specific pieces of information as arguments via their semantic annotations or themes (lookup), and, subsequently, investigate information associated with the argument as per the underlying argument theory (browsing), which associates an argument with its supporting evidence (Warrants, Backing), further details (Elaborations), rebuttals or exceptions (Exceptions), and qualifiers.

In our work, we adapted and extended Argument Theory into a novel Extended Model of Argument (EMA). We applied semantics-based knowledge engineering [14], [15], [16], [17], [18] to engineer an EMA ontology together with an EMA knowledge modeling process, in order to semantically encode PEM for education-oriented dialogue systems. We will show that the resultant (a) is an explorable argument network that, by design, offers credible dialogues over educational material, which (b) supports two types of interactivity: inquiry, i.e., learning via the browsing of educational material, and information-seeking, i.e., ad-hoc coping with situations via searching [19]. We applied our approach to develop a dialogue-based education system for Juvenile Idiopathic Arthritis (JIA) patients and their families. Families have reported feeling overwhelmed by the volume of JIA-related information [20]: dialogue-based education systems can provide a way for them to more effectively interact with PEM. We performed a qualitative evaluation of JADE with JIA health experts. We have made our EMA ontology available online as a contribution to knowledge representation for dialogue systems [21].

Section snippets

Health-related educational dialogue systems

According to a recent survey from 2020 [3], the use of dialogue systems in healthcare is still in its infancy. In general, education-oriented dialogue systems allow the user to manually choose from multiple options [4], [6], [7], [9], [10], [13], [22]; or write the option in free text, which is interpreted using Natural Language Processing (NLP) [7], [8], [11]. In both cases, a dialogue manager matches the user’s questions against a set of pre-defined education content to generate a response.

Using the Toulmin model of argument to computerize PEM

In this section, we illustrate the use of the original TMA to structure Patient Educational Material (PEM). The following is a typical piece of Patient Educational Material (PEM): “If using heat and your child’s skin turns red, although some pinking of the skin is normal, then remove the heat source since this may be a sign of skin damage”. Fig. 1 illustrates the structure of a PEM argument:

Using TMA, we can convey all information in terms of a Claim (“removing the heat source”) and Data on

Results

The CW and SSI were conducted for an average of 26 min (range: 16–37 min). Saturation was achieved, as no new themes were added after the third participant evaluation. Six sub-themes were inductively identified during analysis. Table 2 shows the participant quotes captured during the CW and SSI as grouped per theme. Below, we summarize the participant feedback within the identified sub-theme and indicate the relevant O’Grady category (if any).

Theme 1: Positive responses to the JADE system

Conclusions and future work

We believe that patients can better understand their health condition if they can have an interactive and intelligent dialogue over PEM. A dialogue system can provide such conversational facility, but we found that current work lacks a formal dialogue model to capture credibility, offer multiple types of interactivity, and formulate a salient dialogue without requiring pre-defined conversational paths.

In this paper, we presented the computerization of a formal dialogue model—i.e., Toulmin’s

CRediT authorship contribution statement

Benjamin Rose-Davis: Conceptualization, Methodology, Resources, Investigation, Formal analysis. William Van Woensel: Resources, Software, Writing - original draft, Writing - review & editing, Visualization. Samina Raza Abidi: Supervision, Methodology. Elizabeth Stringer: Supervision, Resources. Syed Sibte Raza Abidi: Conceptualization, Supervision, Methodology, Writing - original draft, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The authors thank the healthcare providers at the IWK Pediatric Rheumatology Clinic at the IWK Health Centre (Halifax, Canada) for their collaboration to the qualitative study.

Ethics Approval

This study was approved by the IWK Research Ethics Board (file number: 1023261).

References (46)

  • J. Mackay et al.

    Online Patient Education and Risk Assessment: project OPERA from Cancerbackup

    Patient Educ. Couns.

    (Aug. 2007)
  • C. Wilson

    “Cognitive Walkthrough”, in User Interface Inspection Methods

    Elsevier

    (2014)
  • E.A.H. Wilson et al.

    Working memory and the design of health materials: A cognitive factors perspective

    Patient Educ. Counseling

    (Mar. 2009)
  • J. Wright

    Parental Perspectives about Research and Knowledge Translation in Juvenile Idiopathic Arthritis

    ACR Open Rheumatol.

    (Mar. 2020)
  • L. Tudor Car

    Conversational Agents in Health Care: Scoping Review and Conceptual Analysis

    J. Med. Internet Res.

    (2020)
  • W.G. Harless et al.

    An Assessment of the Virtual Conversations Method for Prostate Cancer Patient Education

    Urologic Nurs.

    (2007)
  • K.K. Fitzpatrick et al.

    Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial

    JMIR Mental Health

    (Jun. 2017)
  • A. Shamekhi, T. Bickmore, A. Lestoquoy, and P. Gardiner, “Augmenting Group Medical Visits with Conversational Agents...
  • D. Meeker et al.

    SimCoach Evaluation: A Virtual Human Intervention to Encourage Service-Member Help-Seeking for Posttraumatic Stress Disorder and Depression

    (2015)
  • S. Abidi and S. Abidi, “An Ontology-Driven Personalization Framework for Designing Theory-Driven Self-management...
  • S. S. R. Abidi and S. Shayegani, “Modeling the Form and Function of Clinical Practice Guidelines: An Ontological Model...
  • W. Van Woensel, P. C. Roy, S. R. Abidi, and S. S. R. Abidi, “A Mobile and Intelligent Patient Diary for Chronic Disease...
  • W. H. Baig, S. R. Abidi, and S. S. R. Abidi, “An Ontological Model of Behaviour Theory to Generate Personalized Action...
  • Cited by (5)

    • Explanations of Symbolic Reasoning to Effect Patient Persuasion and Education

      2024, Communications in Computer and Information Science
    • Role of HMI in the drug manufacturing process

      2023, Human Machine Interface: Making Healthcare Digital
    • Explainable Clinical Decision Support: Towards Patient-Facing Explanations for Education and Long-Term Behavior Change

      2022, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    View full text