Skip to main content
SpringerLink
Log in

Video based actigraphy and breathing monitoring from the bedside table of shared beds

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Good sleep is an important factor for a high quality of life. Presence of sleep disorders requires patients to undergo polysomnography examinations at a sleep clinic, which involves attaching multiple head and body sensors. This results in an uncomfortable and unnatural sleep setting for the patients, complicating an accurate diagnosis. Currently, this diagnosis is done manually, making the entire process time consuming and cumbersome. We propose a camera based system capable of analyzing the subjects in their own sleeping environment. The system segments the primary subject from the background and any other bed occupants, then computes the actigraphy and the breathing characteristics of the subject. Segmentation is performed with an AdaBoost classifier using among others motion, intensity, focus and Histogram of Oriented Gradients features. A sum of absolute differences operation on the pixels within the segmented area of the primary subject returns the actigraphy signal. The breathing characteristics are extracted based on small motion analysis of consecutive and reference frames. The proposed system has been evaluated on four healthy adults for actigraphy and on five healthy adults for breathing analysis using a Texas Instrument Chronos wrist watch and inductive respiratory belts as references respectively. Evaluation was performed using the metrics accuracy, precision, sensitivity and breathing rate correspondence. The proposed system has an average accuracy of 88 % at a precision of 79 % for segmentation of the primary subject. It can detect movements up to an accuracy of 85 % while outperforming wrist actigraphy at 75 % accuracy. State-of-the-art video based breathing algorithms were surpassed with an overall sensitivity of 87 %, precision of 90 %, and a breathing rate correspondence of 93 %.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Note that high variations in object reflectance for the used IR light makes this classifier weak.

References

  • Allen RP, Hening WA (2009) Actigraph assessment of periodic leg movements and restless legs syndrome. Restless legs syndrome. W.B. Saunders, Philadelphia, pp 142–149

    Chapter  Google Scholar 

  • Ancoli-Israel S, Cole R, Alessi C, Chambers M, Moorcroft W, Pollak CP (2003) The role of actigraphy in the study of sleep and circadian rhythms. Sleep 26(3):342–392

    Google Scholar 

  • Aoki H, Koshiji K, Nakamura H, Takemura Y, Nakajima M (2005) Study on respiration monitoring method using near-infrared multiple slit-lights projection. In: Proceedings of micro-nanomechatronics and human science, IEEE international symposium on 2005, pp 291–296

  • Aubert X, Brauers A (2008) Estimation of vital signs in bed from a single unobtrusive mechanical sensor: algorithms and real-life evaluation. In: Proceedings of 30th annual international conference of the IEEE engineering in Medicine and Biology Society, EMBS 2008

  • Bak JU, Giakoumidis N, Kim G, Dong H, Mavridis N (2012) An intelligent sensing system for sleep motion and stage analysis. In: Procedia Engineering, International Symposium on Robotics and Intelligent Sensors 2012, vol 41(0), pp 1128–1134 (IRIS 2012)

  • Bartula M, Tigges T, Muehlsteff J (2013) Camera-based system for contactless monitoring of respiration. In: Proceedings of Engineering in Medicine and Biology Society (EMBC), 35th annual international conference of the IEEE, pp 2672–2675

  • Beddit (2014) Beddit. http://www.beddit.com/. [Online; accessed 24-May-2014]

  • Coughlin SS, Trock B, Criqui MH, Pickle LW, Browner D, Tefft MC (1992) The logistic modeling of sensitivity, specificity, and predictive value of a diagnostic test. J Clin Epidemiol 45(1):1–7

    Article  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of CVPR, pp 886–893

  • De Chazal P, Fox N, O’Hare E, Heneghan C, Zaffaroni A, Boyle P, Smith S, O’Connell C, McNicholas WT (2011) Sleep/wake measurement using a non-contact biomotion sensor. J Sleep Res 20(2):356–366

    Article  Google Scholar 

  • Frigola M, Amat J, Pagès J (2002) Vision based respiratory monitoring system. In: Proceedings of the 10th Mediterranean Conference on Control and Automation—MED2002 Lisbon, Portugal

  • Fry JM, DiPhillipo MA, Curran K, Goldberg R, Baran AS (1998) Full polysomnography in the home. Sleep 21(6):635–642

    Google Scholar 

  • Gardner R, Grossman W (1976) Normal motor patterns in sleep in man. Adv Sleep Res 2:67–107

    Google Scholar 

  • Gear4 (2014) Renew SleepClock. http://www.stage.gear4.com/product/_/426/renew-sleepclock/. [Online; accessed 24-May-2014]

  • Gederi E, Clifford G (2012) Fusion of image and signal processing for the detection of obstructive sleep apnea. In: Proceedings of biomedical and health informatics (BHI), EEE-EMBS International Conference on 2012, pp 890–893

  • Gonzalez RC, Woods RE (1992) Digital image processing, 2nd edn. Addison-Wesley Longman Publishing Co., Inc, Boston

    Google Scholar 

  • Gori S, Ficca G, Giganti F, Di Nasso I, Murri L, Salzarulo P (2004) Body movements during night sleep in healthy elderly subjects and their relationships with sleep stages. Brain Res Bull 63(5):393–397

    Article  Google Scholar 

  • Haan de G, Biezen PW (1994) Sub-pixel motion estimation with 3-D recursive search block-matching. Signal Process Image Commun 6(3):229–239

    Article  Google Scholar 

  • Harada T, Sakata A, Mori T, Sato T (2000) Sensor pillow system: monitoring respiration and body movement in sleep. In: Proceedings of 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol 1, pp 351–356

  • Heinrich A, Aubert X, de Haan G (2013) Body movement analysis during sleep based on video motion estimation. In: Proceedings of e-health networking applications and services (Healthcom), 15th IEEE international conference on 2013

  • Heinrich A et al (2011) Optimization of hierarchical 3DRS motion estimators for picture rate conversion. IEEE J Sel Top Signal Process 5(2):262–274

    Article  Google Scholar 

  • Horn BK, Schunck BG (1981) Determining optical flow. Artif Intell 17(13):185–203

    Article  Google Scholar 

  • Kalitzin S, Petkov G, Velis D, Vledder B (2012) Automatic segmentation of episodes containing epileptic clonic seizures in video sequences. IEEE Trans Biomed Eng 59(12):3379–3385

    Article  Google Scholar 

  • Karayiannis NB, Varughese B, Tao G, Frost JD Jr, Wise MS, Mizrahi EM (2005) Quantifying motion in video recordings of neonatal seizures by regularized optical flow methods. Trans Image Process 14(7):890–903

    Article  Google Scholar 

  • Karlen W, Mattiussi C, Floreano D (2008) Improving actigraph sleep/wake classification with cardio-respiratory signals. Conf Proc IEEE Eng Med Biol Soc 2008:5262–5265

    Google Scholar 

  • Lagae L, Ceulemans B, Huffel SV, Vanrumste B (2010) Automatic video detection of body movement during sleep based on optical flow in pediatric patients with epilepsy. Med Biol Eng Comput 48:923–931

    Article  Google Scholar 

  • Liao WH, Kuo J (2013) Sleep monitoring system in real bedroom environment using texture-based background modeling approaches. J Ambient Intell Humaniz Comput 4(1):57–66

    Article  Google Scholar 

  • Liao WH, Yang CM (2008) Video-based activity and movement pattern analysis in overnight sleep studies. In: International Conference on Pattern Recognition, pp 1–4

  • Lindh W, Pooler M, Tamparo C (2001) Delmar’s comprehensive medical assisting: administrative and clinical competencies. Delmar Thomson Learning

  • Littner M, Kushida CA, Anderson WM, Bailey D, Berry RB, Davila DG, Hirshkowitz M, Kapen S, Kramer M, Loube D, Wise M, Johnson SF (2003) Practice parameters for the role of actigraphy in the study of sleep and circadian rhythms: an update for 2002. Sleep 26(3):337–341

    Google Scholar 

  • Long X, Foussier J, Fonseca P, Haakma R, Aarts RM (2013) Respiration amplitude analysis for rem and nrem sleep classification. In: Proceedings of Engineering in Medicine and Biology Society (EMBC), 35th annual international conference of the IEEE, pp 5017–5020

  • Muzet A, Naitoh P, Johnson L, Townsend R (1972) Body movements during sleep as a predictor of state change. Psychon Sci 29:7–10

    Article  Google Scholar 

  • Nakajima K, Osa A, Maekawa T, Miike H (1997) Evaluation of body motion by optical flow analysis. Jpn J Appl Phys 36(Part 1, No. 5A):2929–2937

    Article  Google Scholar 

  • Nakajima K, Matsumoto Y, Tamura T (2001) Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed. Physiol Meas 22:21–28

    Article  Google Scholar 

  • National Sleep Foundation (2005) Sleep in America poll: Adult sleep habits and styles. http://www.sleepfoundation.org/article/sleep-america-polls/2005-adult-sleep-habits-and-styles. [Online; accessed 5-October-2013]

  • Omron (2014) Sleepmeter HSL-101 and HSL-001 SleepDesign Lite. http://www.healthcare.omron.co.jp/product/hsl/hsl-101.html. [Online; accessed 24-May-2014]

  • Samy L, Huang MC, Liu J, Xu W, Sarrafzadeh M (2014) Unobtrusive sleep stage identification using a pressure-sensitive bed sheet. IEEE Sens J 14(7):2092–2101. doi:10.1109/JSEN.2013.2293917

    Article  Google Scholar 

  • Stigler S (1989) Francis galton’s account of the invention of correlation. Stat Sci 4(2):73–79

    Article  MATH  MathSciNet  Google Scholar 

  • Tan KS, Saatchi R, Elphick H, Burke D (2010) Real-time vision based respiration monitoring system. In: Proceedings of communication systems networks and digital signal processing (CSNDSP), 7th international symposium on 2010, pp 770–774

  • TMSI (2013) Respiration effort module v6 and inductive respiration band, adult l. http://www.tmsi.com/images/stories/PDF-files/Accessorylist_2013.pdf. [Online; accessed 5-October-2013]

  • Tsai DM, Wang HJ (1998) Segmenting focused objects in complex visual images. Pattern Recognit Lett 19(10):929–940

    Article  Google Scholar 

  • Vink JP, de Haan G (2012) Comparison of machine learning techniques for target detection. Artif Intel Rev 1–15. doi:10.1007/s10462-012-9366-7

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings computer vision and pattern recognition, CVPR 2001, IEEE Computer Society Conference on 2001, vol 1, pp I-511–I-518

  • Wang CW, Ahmed A, Hunter A (2006) Vision analysis in detecting abnormal breathing activity in application to diagnosis of obstructive sleep apnoea. Conf Proc IEEE Eng Med Biol Soc 1:4469–4473

    Article  Google Scholar 

  • Wang CW, Hunter A, Gravill N, Matusiewicz S (2010) Real time pose recognition of covered human for diagnosis of sleep apnoea. Comput Med Imaging Graph 34(6):523–533

    Article  MATH  Google Scholar 

  • Wilde-Frenz J, Schulz H (1983) Rate and distribution of body movements during sleep in humans. Percept Mot Skills 56:275–283

    Article  Google Scholar 

  • Withings (2014) Withings Aura Smart Sleep System. http://withings.com/en/aura. [Online; accessed 24-May-2014]

  • Yang FC, Kuo CH, Tsai MY, Huang SC (2003) Image-based sleep motion recognition using artificial neural networks. In: Proceedings of machine learning and cybernetics, international conference on 2003, vol 5, pp 2775–2780

  • Yushkevich P (2012) Itk-snap. http://www.itksnap.org. [Online; accessed 5-October-2013]

Download references

Author information

Authors and Affiliations

  1. Philips Research Laboratories, High Tech Campus 36, 5656 AE, Eindhoven, The Netherlands

    Adrienne Heinrich, Frank van Heesch & Mukul Rocque

  2. Intel Corporation, High Tech Campus 83, 5656 AG, Eindhoven, The Netherlands

    Bhargava Puvvula

Authors
  1. Adrienne Heinrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank van Heesch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bhargava Puvvula
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mukul Rocque
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adrienne Heinrich.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Heinrich, A., van Heesch, F., Puvvula, B. et al. Video based actigraphy and breathing monitoring from the bedside table of shared beds. J Ambient Intell Human Comput 6, 107–120 (2015). https://doi.org/10.1007/s12652-014-0247-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-014-0247-6

Keywords

Search

Navigation