Keywords

1 Introduction

Smart watches have seen rapid and widespread adoption by consumers in the past few years, with an expected market demand for these devices reaching up to 214 million units in 2018 [1]. These network-enabled, wrist-worn devices represent an unprecedented opportunity to support improved patient self-management and health monitoring in everyday life through an array of on-board sensors, computing capability, and communication features. Indeed, two recent systematic reviews found smart watch studies targeting numerous health-related applications have appeared at an increasing rate in the scientific literature in a few short years [2, 3].

The objective of this paper is to illustrate potential uses and challenges of smart watches for health-related decision making by describing these devices in relation to an Augmented Cognition (AugCog) research approach. We first conduct a brief review of AugCog research, relate AugCog research to that of Clinical Decision Support (CDS) research in the health domain, and discuss smart watch applications based on prior research with these devices. We then outline potential augmented cognition research directions in the health domain using smart watches by describing sensing modalities and potential mitigation strategies in relation to Persons Living With HIV (PLWH) [4,5,6,7], Chronic Obstructive Pulmonary Disorder patients [8,9,10,11,12], and emerging patient-centered workflow paradigms [13,14,15,16,17,18].

1.1 Augmented Cognition

The goal of AugCog is to enhance end user cognitive capacity and capability in support of human task performance via automated adaption of technical system function and information presentation in a closed loop system [19, 20]. Three principal AugCog research areas are: Cognitive State Assessment (CSA) enabled by sensor-based capture of cognitive or functional state; Mitigation Strategies (MS) that respond to cognitive state through closed-loop system adjustments; and Robust Controllers (RC) that allow systems to function with resilience under diverse operating conditions [20, 21].

1.2 Clinical Decision Support

A clinical decision support (CDS) system is “designed to be a direct aid to clinical decision making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician or the patient for a decision” [22]. Prior CDS research has identified a rank-ordered list of ten grand challenges to improve the design of CDS systems [23]. Four of these ten grand challenges have been designated as necessary to improve the effectiveness of CDS interventions; the remaining six pertain to creation of new CDS interventions and dissemination of existing CDS knowledge.

The four grand challenges for improved effectiveness of CDS interventions are ranked as follows: Improve the human–computer interface (first out ten); Summarize patient-level information (third out of ten); Prioritize and filter recommendations (fourth out of ten) and; Synthesize recommendations for comorbidities (sixth out of ten) [23]. These four grand challenges are relevant to two cognitive bottlenecks identified from AugCog research [24]. Table 1 displays these four CDS grand challenges [23] with the associated cognitive bottleneck and the AugCog approach to overcome the bottleneck [24]. Narrow user input capabilities refer to limitations of system designs that present barriers to information entry. Information overload refers to the inability of users to process vast amounts of system output. A cognitive state sensor “acquires physiological and behavioral parameter(s) that can be reliably associated with specific cognitive states, which can be measured in real time while an individual or team of individuals is engaged with a system” [21].

Table 1. CDS grand challenges and AugCog approaches to overcome cognitive bottlenecks
Full size table

1.3 Linking Augmented Cognition and Clinical Decision Support

Notably, the top-ranked CDS grand challenge of Improve the human–computer interface [23] represents the greatest opportunity for AugCog integration with CDS systems. Figure 1 illustrates how the concept of a CDS system aligns with the Fundamental Theorem of Biomedical Informatics, which states: “A person working in partnership with an information resource is ‘better’ than that same person unassisted” [25].

Fig. 1.
figure 1

Graphic demonstrating the fundamental theorem of biomedical informatics [25]

Full size image

In describing seminal AugCog efforts, Schmorrow and Kruse state:

AugCog will enable computational systems to adapt to the user, rather than forcing the user to adapt to the computational systems. In this way the AugCog program moves beyond the traditional approach to redesigning human-computer interfaces - which often fail to take the state of the user into account [19].

Taken together, these statements illustrate that the AugCog approach of cognitive state assessment using a cognitive state sensor is complementary to the notion of a CDS system that aligns with the Fundamental Theorem of Biomedical Informatics. However, the health domain represents a context with stakeholders who may have conflicting goals (e.g.: patients, family members, and health care providers with different roles). As a result, there may be fewer parallels between the health domain of patients in everyday life, the military domain where AugCog research originated [19, 21], and nuclear power plant control rooms where AugCog approaches have translated [20]. In particular, “operators” in military and control room environments possess specialized training, skills, and protocols designed to facilitate achievement of organizational objectives whereas patients in everyday life may not. Still, there have been recent AugCog forays into the health research domain [26,27,28]. In addition, while there is a large body of clinical decision support research, much of it is focused on health care provider decision making in clinical contexts using data from electronic health records [22, 23, 29]. Therefore, advances in smart watch technology represent new opportunities to support health-related decision making for patients in everyday life using AugCog and CDS approaches. Our proposed health research agenda will expand the types of sensors and sensor data for AugCog purposes, beyond those of physiological data, in a similar way that others have already begun [28].

2 Sensors in Health Research

Sensors have long been posited as a means to support health-related activities outside clinical care settings through incorporation of sensor data in health applications. Some common sensor types in health studies include smart home sensor technologies installed in the residential environment that enable passive monitoring of health [30,31,32,33] and wearable technologies [34, 35]. Each has its own trade-offs in terms of technology function and acceptance. For instance, smart homes require no user interaction beyond initial agreement to install the technology yet only collect data when a person is home and have difficulty distinguishing between multiple residents [31]. Wearable technologies can enable activity data collection anywhere, matched to an individual, but can present adherence issues if technologies are not worn properly [34].

2.1 Smart Watches in Health Research

Smart watches are a relatively new innovation in wearable technology. That being said, numerous smart watch health studies have been conducted since 2014 [2, 3]. While research with these wearable devices is still at a nascent stage, the purposes for which they have been put to use vary widely by sensing modality and study focus. Smart watches have been used for detection of activity levels, emotional state, seizures, tremor, posture, heart rate, temperature, speech therapy progress, and eating, medication-taking, and scratching behaviors [2]. These studies have employed a variety of onboard sensing modalities including accelerometers, gyroscopes, microphones, optical sensors, contact sensors, ambient light sensors, and received signal strength indication (RSSI) localization. Smart watch applications have been primarily used during proof-of concept studies to determine if smart watches are feasible for research. Studies that enrolled persons living with targeted conditions were few and focused on Parkinson’s disease, epilepsy, and diabetes management.

2.2 Rapid Technology Change as a Challenge

In a previous AugCog paper focused on technology-supported health measures for congestive heart failure (CHF) patients [36], we described the Lab of Things platform [37, 38] and Pebble smart watches (http://www.getpebble.com) as promising technologies for integration in the home. However, rapid changes due to market forces and business initiatives can cause technologies to become unavailable or unsupported. As of 2016, the Pebble smart watch company went out of business and the Lab of Things [37, 38] Internet of Things platform is no longer supported by Microsoft Research.

3 Future Research

3.1 Patient Cases

3.1.1 Smart Watch Potential to Support AugCog

Prior smart watch research shows promise to support specific needs for PLWH and COPD patients. Table 2 illustrates smart watch sensor types for cognitive state assessment, mitigation strategies these sensors may support, and the types of studies needed to develop robust controllers for health-related AugCog.

Table 2. Technologies and recommendations for AugCog health research using smart watches
Full size table

3.1.2 Persons Living with HIV (PLWH)

PLWH must manage everyday behaviors related to antiretroviral medication use, which requires a higher level of adherence than most drugs, and prevention behaviors, which protect them from other STDs and reduce the chances that they will spread HIV to others. These behaviors can be hard to maintain, even with the best of intentions. Mobile technology has proven effective in improving antiretroviral therapy (ART) adherence through daily tailored messages [4,5,6,7]. Future research could extend these findings to more immediate messaging delivered via smart watches, use of haptic cues as reminders to take medication, or geolocation to deliver specific messages at specific times when PLWH are in higher-risk environments. Mobile cues to engage in other healthy behaviors are also particularly relevant to PLWH, who have higher rates of smoking, alcohol, and drug use than the general population, and must manage nutrition and exercise to counteract medication-related increases in cardiovascular risk. Table 2 shows smart watch potential to support PLWH.

3.1.3 Chronic Obstructive Pulmonary Disorder (COPD)

COPD is a chronic progressive disease that requires the individual to monitor their breathing, daily activity patterns, and self-manage medications such as metered dose inhalers (MDIs) [8]. There are multiple steps to effective deliver of MDI medications with individuals with COPD, only successful about half of the time. The interplay of these behaviors requires significant cognitive skills such as sensation, temperature, humidity, and air quality monitoring and appraisal within the overlay of daily activity expectations and successful medication deliver. Individuals with COPD do have some cognitive challenges in managing their health and well-being and the stress and anxiety of being short of breath while processing multiple sources of information to determine whether to take action or simply rest and continue monitoring would totally benefit from AugCog [8,9,10,11,12]. Providing summary information, monitoring physical cues such as heart rate or respiratory rate as well as mobile cues as to environmental concerns would move COPD self-management forward and is the next frontier for health care-focused mobile technology research. Table 2 shows smart watch potential to support COPD.

3.2 Types of Studies Needed

3.2.1 Workflow

Workflow research, which traditionally focuses on activities and temporal relationships between them, can contribute to understanding of the relationship between AugCog and health-related decision making, and also inform the design, implementation and evaluation of technology-based interventions that support decision making [14, 16]. Workflow research typically yields three types of deliverables: Rich workflow descriptions (narratives); quantitative representations (e.g. Petri Nets or Markov Chains); and visual depictions (e.g. workflow diagrams) [14]. These three deliverables are beneficial in the examination of the relationships between AugCog and decision making at a high-level as described in Table 1. Workflow research can be particularly helpful in highlighting temporal relationships that translate correlations into causal relationships about how AugCog approaches can improve decision making. The informatics literature shows that if workflows that represent current practice are not accounted for in the design and implementation of new systems, technology changes can disrupt workflow in intended and unintended ways and leads to poor performance and outcomes [48]. Thus, the effect of AugCog research on decision making must take workflow into account to realize the full potential of AugCog interventions. One gap in current workflow research is that field studies typically focus on behavioral indicators without regard to cognitive activities. Cognitive activities are either ignored or they are assumed. The ability to understand the role of cognitive activities as an underlying feature of behavioral indicators can allow researchers to examine workflow and cognition in the temporal context of health-related activities as whole. Smart watches can play a critical role in narrowing this gap by providing by providing real-time, in situ assessment of cognitive state through onboard sensors and communication features.

3.2.2 Technology Function and Acceptability

Studies must be conducted to validate and understand behaviors from data collected by smart watches. Smart watch technical function must be validated through comparison studies using known research-grade devices as the gold standard. This research should also explore strategies to overcome the challenge of device and platform obsolescence due to rapid change in the technology landscape [49]. Studies that develop new methods to estimate behavior from activity data, and other types of data, will be required on a continuous basis as more and different types of sensor-based measurements become available [50,51,52,53]. Important to understand are the factors related to why a given individual will use or abandon a wearable device like a smart watch [54]. Not surprisingly, smart watch acceptance research is at an early stage due to recent availability of smart watches as a consumer-grade device [55,56,57,58,59].

3.2.3 Informatics Study Types and Task Complexity

Friedman et al. define a set of informatics study classifications that describe the range of what can be studied based on type of research question [60]. Studies are classified as: needs assessment (“what is the problem?”); design validation (“is the development method appropriate?”); structure validation (“does the system function as intended?”); usability test (“can targeted stakeholders use it as intended?”); laboratory function (“does it have potential for benefit?”); field function (“does it have the potential for benefit in the real world?”); laboratory user effect (“is it likely to change behavior?”); field user effect (“does it change behavior in the real world?”); problem impact (“does it have a positive impact on the defined problem?”) [60]. These informatics study types are non-exclusive and should be conducted iteratively using smart watches to support health-related decision making for PLWH and COPD.

These studies should be designed with the aim of providing task advantages that deliver real-time resources to meet the personal goals of PLWH and COPD patients [36]. Prior research on task complexity [61] can inform the design and conduct of these studies using smart watches as an integral data collection and communication device. For example, the first Apple Watch applications for COPD currently do not gather or summarize the appropriate information and coordinate this with reminders or alarms that allow the patient to fully self-manage their condition. Patients must manually self-monitor but these data should be seamlessly integrated with environmental conditions, current emotional state, and complexity of medication regimes. At present, these smart watch applications have heavy input requirements with little synthesis or summary of material in ways that patients could use to modify activities, take preventive actions or address a current physical sensation.

4 Conclusion

In this paper, we have reviewed current health-related research using smart watches in terms of an Augmented Cognition research agenda. In doing so, we have connected this AugCog research agenda with prior Clinical Decision Support, workflow, and health informatics research using Persons Living With HIV (PLWH) and Chronic Obstructive Pulmonary Disorder (COPD) as examples of future AugCog research directions for health-related decision making.

Chronic disease management and health care domain areas differ greatly from the original AugCog military research domain. However, AugCog research approaches have great potential to facilitate improved decision making, self-management and health outcomes for PLWH, COPD, and others, by using information collected over the long-term. One potential way to improve patient self-management and health outcomes is through automated and passive collection of information about activity levels, emotions, risky situations, or other health behaviors from smart watches to support augmented cognition for the wearer. However, before that happens, interdisciplinary researchers must understand where AugCog meshes with these domains and identify potential advantages of smart watch information for patients, family members, and health care providers. Ultimately, the goal is to enable community-based field studies that enroll persons living with the conditions targeted by smart watch interventions.