Research paper
Reliability of commercially available sleep and activity trackers with manual switch-to-sleep mode activation in free-living healthy individuals

https://doi.org/10.1016/j.ijmedinf.2017.03.008Get rights and content

Highlights

  • Sleep and activity monitors are reliable for total sleep time measurements, but not for sleep architecture assessment.

  • Monitors with switch-to-sleep mode activation (Up Move Jawbone® and Withings Pulse 02®) do not improve performances.

  • As we do not know about the algorithms of these monitors, we could not perform detailed analyzes.

  • A partnership between researchers and providers could result in the construction of better working algorithms.

Abstract

Introduction

Wearable health devices have become trendy among consumers, but it is not known whether they accurately measure sleep and physical activity parameters. To address this question, we have studied the measured data of two consumer-level activity monitors (Up Move Jawbone® (U) and Withings Pulse 02® (W)) and compared it with reference methods for sleep and activity recordings, namely the Bodymedia SenseWear Pro Armband® actigraph (SWA) and home-polysomnography (H-PSG).

Methods

Twenty healthy patients were assessed at home, during sleep, with the four devices. An additional 24-h period of recording was then planned during which they wore the 2 trackers and the SWA. Physical activity and sleep parameters obtained with the 4 devices were analyzed.

Results

Significant correlations with H-PSG were obtained for total sleep time (TST) for all the devices: r = 0.48 for W (p = 0.04), r = 0.63 for U (p = 0.002), r = 0.7 for SWA (p = 0.0003). The best coefficient was obtained with SWA. Significant correlations were also obtained for time in bed (TIB) for U and SWA vs PSG (r = 0.79 and r = 0.76, p < 0.0001 for both) but not for W (r = 0.45, p = 0.07). No significant correlations were obtained for deep sleep, light sleep, and sleep efficiency (SE) measurements with W, U and SWA. Sleep latency (SL) correlated with H-PSG only when measured against SWA (r = 0.5, p = 0.02).

Physical activity assessment revealed significant correlations for U and W with SWA for step count (both r = 0.95 and p < 0.0001) and active energy expenditure (EE) (r = 0.65 and 0.54; p = 0.0006 and p < 0.0001). Total EE was also correctly estimated (r = 0.75 and 0.52; p < 0.0001 and p = 0.001).

Conclusion

Sleep and activity monitors are only able to produce a limited set of reliable measurements, such as TST, step count, and active EE, with a preference for U which performs globally better. Despite the manual activation to sleep mode, U and W were not suitable for giving correct data such as sleep architecture, SE, and SL. In the future, to enhance accuracy of such monitors, researchers and providers have to collaborate to write algorithms based reliably on sleep physiology. It could avoid misleading the consumer.

Introduction

During the last few years, numerous novel wearable health technology devices, such activity and sleep monitors, have been commercialized. These devices are simple, appealing, and inexpensive and provide an opportunity for anyone to obtain feedback on their personal daytime and nighttime behavior via mobile apps running on smartphones. These devices operate on accelerometer-based technology, but little is known about their true ability to accurately measure sleep parameters and physical activity.

To date, the reference method for sleep recording is polysomnography (PSG) [1], but the technique is expensive, complex, and time-consuming and, for these reasons, researchers have developed simplified devices to assess sleep parameters and sleep schedules. Wrist-worn actigraphs are used routinely in ambulatory patients and are able to differentiate sleep and wake times, and to assess sleep quantity and sleep schedules. They have been validated against PSG and have shown good agreement in healthy subjects [2]. They are routinely used for assessment of sleep disorders, such as circadian rhythm disorders and insomnia [3]. These devices are also able to accurately measure energy expenditure and physical activity, for periods as long as 14 days [4].

The purpose of our study was to compare the accuracy of two different consumer-level activity and sleep trackers with manual switch-to-sleep mode activation with validated tools (PSG and actigraphy) to measure sleep parameters and physical activity in healthy subjects.

Section snippets

Study design

In this prospective study, we evaluated healthy patients for 36 h, during both sleep and daily active periods. The study started with one night of recording. Home-polysomnography (H-PSG) was performed simultaneously with 3 other recordings: 2 consumer-level activity and sleep monitors (Up Move Jawbone® and Withings Pulse 02®) and one actigraph (Bodymedia SenseWear Pro Armband®). The morning after, H-PSG was disconnected and patients were asked to keep the 3 other devices on for an additional 24 h.

Results

Twenty patients (7 men) were recorded. Mean age was 30 ± 5 years.

Discussion

In this small series of healthy patients recorded concomitantly by two commercially available activity and sleep monitors and two validated tools (PSG and SWA), we observed that wearable app-based health technologies give limited information regarding sleep structure and activity. We confirm the narrow performance of such monitors, which is not improved by the use of devices with manual activation to sleep mode. Monitors continue to grow in popularity and there is a risk of misleading

Conflict of interests

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Authors’ contributions

A. Gruwez, W. Libert, L. M. Bruyneel have collected the data, performed data analyses and prepared the manuscript.

L. Ameye have performed the statistical analyses and revised the manuscript.

Summary Points

What was already known before this study?

  • Sleep and activity monitors, without switch-to-sleep mode activation, are reliable for TST measurements, but not for SE, TIB, SL and sleep architecture assessment.

  • Some studies suggested that the two activity and sleep trackers with switch-to-sleep mode

Acknowledgements

The authors would like to acknowledge the contribution of a medical writer, Sandy Field, PhD, for formatting and English language editing of this manuscript.

References (23)

  • M. Shin et al.

    The validity and Actiwatch2 and sensewear armband compared against polysomnography at different ambient temperature conditions

    Sleep Sci.

    (2015)
  • A.S. Brazeau et al.

    Validation and Reliability of two activity monitors for energy expenditure assessment

    J. Sci. Med. Sport

    (2016)
  • C.A. Kushida et al.

    Practice Parameters for the indications for polysomnography and related procedures: an update for 2005

    Sleep

    (2005)
  • A.T.M. Van de Water et al.

    Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography– a systematic review

    J. Sleep Res.

    (2011)
  • T.I. Morgenthaler et al.

    Practice parameters for the use of autotitrating continuous positive airway pressure devices for titrating pressures and treating adult patients with obstructive sleep apnea syndrome: an update for 2007. An American Academy of Sleep Medicine report

    Sleep

    (2008)
  • D. John et al.

    Comparison of four ActiGraph accelerometers during walking and running

    Med. Sci. Sports Exerc.

    (2010)
  • UPmove a Great Shape to Be in

    (2016)
  • Withings Inspire Health

    (2016)
  • D.M. O'Driscoll et al.

    Energy expenditure in obstructive sleep apnea: validation of a multiple physiological sensor for determination of sleep and wake

    Sleep Breath.

    (2013)
  • M. Schimpl et al.

    Association between walking speed and age in healthy, free-living individuals using mobile accelerometry – a cross-sectional study

    PLoS One

    (2011)
  • I.M. Lee et al.

    Physical activity and weight gain prevention

    JAMA

    (2010)
  • Cited by (36)

    • A systematic review of the accuracy of sleep wearable devices for estimating sleep onset

      2020, Sleep Medicine Reviews
      Citation Excerpt :

      Whilst actigraphy devices have the same underlying rationale that motion equates to sleep/wake state, there may be differences in accuracy between actigraphy models. Thirteen studies used at least two different actigraphy devices simultaneously [30,31,39,43,45,47,51,57,59,70,76,80,81]. Most of these studies have tested Philips Actiwatch devices (10/13 studies), with mean discrepancies ranging from a 2.3-min overestimation to a 14-min underestimation.

    • Wearable activity trackers, accuracy, adoption, acceptance and health impact: A systematic literature review

      2019, Journal of Biomedical Informatics
      Citation Excerpt :

      Fig. 2 suggests that research on the subject of WAT device use for Patient Treatment and Medical Settings is the second most common research theme, and empirical interest in this domain has increased steadily over time (see Fig. 3). Patients’ use of WAT overcomes barriers of conventional health measurements and allows collection of new types of patient monitoring information (e.g., steps [26], heart rate [76], and sleep [152]). This in turn creates opportunities to better manage patients’ functional status assessment and support beyond the clinical setting [21].

    • Combined interventions for physical activity, sleep, and diet using smartphone apps: A scoping literature review

      2019, International Journal of Medical Informatics
      Citation Excerpt :

      These apps have become popular due to their ambient data gathering and analysis [20–22]. For consumers looking to improve their health, these apps facilitate self-monitoring [23] of their activities like step counts, food logs [24] and sleep [25], and offer a way to gain feedback through data-based insights e.g., plots or visualizations of their step counts. Further, as mentioned earlier, interventions need to be personalized to the individual, which could be achieved through user profiles[26].

    View all citing articles on Scopus
    View full text