Skip to main content

Advertisement

Springer Nature Link
Log in

Chest area segmentation in 3D images of sleeping patients

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Although the field of sleep study has greatly developed over recent years, the most common and efficient way to detect sleep issues remains a sleep examination performed in a sleep laboratory. This examination measures several vital signals by polysomnograph during a full night’s sleep using multiple sensors connected to the patient’s body. Nevertheless, despite being the gold standard, the sensors and the unfamiliar environment’s connection inevitably impact the quality of the patient’s sleep and the examination itself. Therefore, with the novel development of accurate and affordable 3D sensing devices, new approaches for non-contact sleep study have emerged. These methods utilize different techniques to extract the same breathing parameters but with contactless methods. However, to enable reliable remote extraction, these methods require accurate identification of the basic region of interest (ROI), i.e., the patient’s chest area. The lack of automated ROI segmenting of 3D time series is currently holding back the development process. We propose an automatic chest area segmentation algorithm that given a time series of 3D frames containing a sleeping patient as input outputs a segmentation image with the pixels that correspond to the chest area. Beyond significantly speeding up the development process of the non-contact methods, accurate automatic segmentation can enable a more precise feature extraction. In addition, further tests of the algorithm on existing data demonstrate its ability to improve the sensitivity of a prior solution that uses manual ROI selection. The approach is on average 46.9% more sensitive with a maximal improvement of 220% when compared to manual ROI. All mentioned can pave the way for placing non-contact algorithms as leading candidates to replace existing traditional methods used today.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WM, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. Chest 124(4):1543–1579 [Online]. Available: http://linkinghub.elsevier.com/retrieve/pii/S0012369216487052

  2. Procházka A, Schätz M, Centonze F, Kuchyňka J, Vyšata O, Vališ M (2016) Extraction of breathing features using MS Kinect for sleep stage detection. Signal Image and Video Processing 10(7):1279–1286

    Article  Google Scholar 

  3. Gibson, GJ (2004) Obstructive sleep apnoea syndrome: underestimated and undertreated. Br Med Bull 72(1):49–64. [Online]. Available: https://academic.oup.com/bmb/article/72/1/49/272854

  4. Al-Naji A, Gibson K, Lee S-H, Chahl J (2017) Real time apnoea monitoring of children using the microsoft kinect sensor: a pilot study. Sensors 17(2):286. [Online]. Available: http://www.mdpi.com/1424-8220/17/2/286

  5. Palmero C, Esquirol J, Bayo V, Cos MÀ, Ahmadmonfared P, Salabert J, Sánchez D, Escalera S (2017) Automatic sleep system recommendation by multi-modal RBG-depth-pressure anthropometric analysis. Int J Comput Vision 122(2):212–227. [Online]. Available: http://link.springer.com/10.1007/s11263-016-0919-0

  6. Schätz M, Procházka A, Kuchyňka J, Vyšata O (2020) Sleep apnea detection with polysomnography and depth sensors. Sensors (Switzerland)

  7. Hirshkowitz M (2014) Polysomnography: understanding this technology’s past might guide future developments. IEEE Pulse 5(5):26–28. [Online]. Available: http://ieeexplore.ieee.org/document/6908144/

  8. Young T, Peppard PE, Gottlieb DJ (2002) Epidemiology of obstructive sleep apnea: a population health perspective, pp 1217–1239

  9. Charlton PH, Birrenkott DA, Bonnici T, Pimentel MAF, Johnson AEW, Alastruey J, Tarassenko L, Watkinson PJ, Beale R, Clifton DA (2018) Breathing rate estimation from the electrocardiogram and photoplethysmogram: a review. IEEE Rev Biomed Eng 11:2–20. [Online]. Available: https://doi.org/10.1109/rbme.2017.2763681

  10. Lazazzera R, Laguna P, Gil E, Carrault G (2021) Proposal for a home sleep monitoring platform employing a smart glove. Sensors 21(23). [Online]. Available: https://www.mdpi.com/1424-8220/21/23/7976

  11. Dafna E, Tarasiuk A, Zigel Y (2015) Sleep-wake evaluation from whole-night non-contact audio recordings of breathing sounds. PLoS ONE 10(2)

  12. Harte JM, Golby CK, Acosta J, Nash EF, Kiraci E, Williams MA, Arvanitis TN, Naidu B (2016) Chest wall motion analysis in healthy volunteers and adults with cystic fibrosis using a novel Kinect-based motion tracking system, Med Biol Eng Comput 54(11):1631–1640. [Online]. Available: http://link.springer.com/10.1007/s11517-015-1433-1

  13. Kagawa M, Ueki K, Tojima H, Matsui T (2013) Noncontact screening system with two microwave radars for the diagnosis of sleep apnea-hypopnea syndrome, Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference 2013:2052–2055

    Google Scholar 

  14. Yang C, Cheung G, Stankovic V, Chan K, Ono N (2017) Sleep apnea detection via depth video and audio feature learning. IEEE Trans Multimed 19(4):822–835

    Article  Google Scholar 

  15. Schätz M, Centonze F, Kuchynka J, Tupa O, Vysata O, Geman O, Prochazka A (2015) Statistical recognition of breathing by MS Kinect depth sensor. In: 2015 International workshop on computational intelligence for multimedia understanding (IWCIM). IEEE, pp 1–4. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7347062

  16. Nguyen P, Transue S, Choi M-H, Halbower AC, Vu T (2016) WiKiSpiro: non-contact respiration volume monitoring during sleep. 8th Wireless of the Students, by the Students, and for the Students Workshop, S3, vol. 03-07-Octo, no 2, pp. 27–29. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84994184650&partnerID=40&md5=5f9db30d9f6744108d158e859642a55e

  17. Carfagni M, Furferi R, Governi L, Servi M, Uccheddu F, Volpe Y (2017) On the performance of the intel sr300 depth camera: metrological and critical characterization. IEEE Sensors J 17(14):4508–4519

    Article  Google Scholar 

  18. Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2012) Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computing 30(3):217–226

    Article  Google Scholar 

  19. Garn H, Kohn B, Dittrich K, Wiesmeyr C, Kloesch G, Stepansky R, Wimmer M, Ipsiroglu O, Grossegger D, Kemethofer M, Seidel S (2016) 3D detection of periodic limb movements in sleep. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 427–430. [Online]. Available: http://ieeexplore.ieee.org/document/7590731/

  20. Chen L, Wei H, Ferryman J (2013) A survey of human motion analysis using depth imagery. Pattern Recogn Lett 34(15):1995–2006

    Article  CAS  Google Scholar 

  21. Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2011) Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow. 2011 IEEE international conference on automatic face and gesture recognition and workshops, FG 2011, pp 700–706

  22. Cao Z, Hidalgo G, Simon T, Wei SE, Sheikh Y (2021) OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(1):172–186

    Article  Google Scholar 

  23. Wang W, Zhu H, Dai J, Pang Y, Shen J, Shao L (2020) Hierarchical human parsing with typed part-relation reasoning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 8929–8939

  24. Wang W, Zhang Z, Qi S, Shen J, Pang Y, Shao L (2019) Learning compositional neural information fusion for human parsing. In: 2019 IEEE/CVF international conference on computer vision (ICCV). pp 5702–5712

  25. Plagemann C, Ganapathi V, Koller D, Thrun S (2010) Real-time identification and localization of body parts from depth images. In: 2010 IEEE international conference on robotics and automation. IEEE, pp 3108–3113

  26. Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In CVPR 2011. IEEE, pp 1297–1304

  27. Okuno A, Ishikawa T, Watanabe H (2020) Rollover detection of infants using posture estimation model. In: 2020 IEEE 9th global conference on consumer electronics (GCCE). IEEE, pp 490–493

  28. Mallick T, Das PP, Majumdar AK (2014) Characterizations of noise in Kinect depth images: a review. pp. 1731–1740

  29. Huang W, Wai AAP, Foo SF, Biswas J, Hsia C-C, Liou K (2010) Multimodal sleeping posture classification. In: 2010 20th International conference on pattern recognition. IEEE, pp 4336–4339

Download references

Acknowledgements

Real data were kindly provided by the Department of Neurology of the Faculty hospital of Charles University in Hradec Kralove. The project was approved by the Local Ethics Committee as stipulated by the Helsinki Declaration.

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, Open University of Israel, Raanana, Israel

    Yoav Goldstein & Mireille Avigal

  2. Department of Computing and Control Engineering, University of Chemistry and Technology, Prague, Czech Republic

    Martin Schätz

Authors
  1. Yoav Goldstein
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Martin Schätz
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mireille Avigal
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yoav Goldstein.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goldstein, Y., Schätz, M. & Avigal, M. Chest area segmentation in 3D images of sleeping patients. Med Biol Eng Comput 60, 2159–2172 (2022). https://doi.org/10.1007/s11517-022-02577-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02577-1

Keywords