Abstract
Computer vision based patient activity monitoring systems can be attractive for various unobtrusive clinical applications. Such a monitoring system can be developed using movement information derived from the skeleton model of the current body pose, e.g. obtained using a depth camera. Earlier research using estimated skeleton models have been focused mostly on gaming applications. In this paper, we propose CNN-SkelPose as a skeleton model estimation method for clinical applications. CNN-SkelPose uses a trained Convolutional Neural Network to extract both the local and global information from the depth image. CNN-SkelPose outperforms the baseline model of Skeltrack for reliable skeleton model estimation in patient monitoring scenarios. Our results show the inadequacy of existing methods for skeleton model estimation when applied to a clinical scenario and suggests CNN-SkelPose as an improvement towards this application.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achilles F, Ichim AE, Coskun H, Tombari F, Noachtar S, Navab N (2016) Patient mocap: Human pose estimation under blanket occlusion for hospital monitoring applications. In: International Conference on Medical Image Computing and Computer—Assisted Intervention, Springer, pp 491–499
Ancoli-Israel S, Cole R, Alessi C, Chambers M, Moorcroft W, Pollak CP (2003) The role of actigraphy in the study of sleep and circadian rhythms. Sleep 26(3):342–392
Baak A, Müller M, Bharaj G, Seidel HP, Theobalt C (2013) A data-driven approach for real-time full body pose reconstruction from a depth camera. In: Consumer Depth Cameras for Computer Vision, Springer, pp 71–98
Banerjee T, Enayati M, Keller JM, Skubic M, Popescu M, Rantz M (2014) Monitoring patients in hospital beds using unobtrusive depth sensors. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp 5904–5907
Belagiannis V, Rupprecht C, Carneiro G, Navab N (2015) Robust optimization for deep regression. In: IEEE Computer Society, Washington, DC, USA, pp 2830–2838
Fook VFS, Thang PV, Htwe QQ, Phyo AAP, Jayachandran BJ, Yap P (2007) Automated recognition of complex agitation behavior of demented patient using video camera. In: 9th International Conference on e-Health Networking, Application and Services
Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp 315–323
Godfrey A, Conway R, Leonard M, Meagher D, OLaighin GM (2008) Motion analysis in delirium: A wavelet based approach for sub classification. In: 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, pp 3574–3577
Godfrey A, Conway R, Leonard M, Dv Meagher, Ólaighin GM (2010) Motion analysis in delirium: a discrete approach in determining physical activity for the purpose of delirium motoric subtyping. Med Eng Phys 32(2):101–110
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press, iSBN: 9780262035613, Page: 226
Heinrich A, Zhao X, de Haan G (2013) Multi-distance motion vector clustering algorithm for video-based sleep analysis. In: e-Health Networking, Applications & Services (Healthcom), 2013 IEEE 15th International Conference on, IEEE, pp 223–227
Heinrich A, Geng D, Znamenskiy D, Vink JP, de Haan G (2014) Robust and sensitive video motion detection for sleep analysis. IEEE J Biomed Health Inf 18(3):790–798
Heinrich A, van Heesch F, Puvvula B, Rocque M (2015) Video based actigraphy and breathing monitoring from the bedside table of shared beds. J Ambient Intell Human Comput 6(1):107–120
Leonard M, Godfrey A, Silberhorn M, Conroy M, Donnelly S, Meagher D, Ólaighin G (2007) Motion analysis in delirium: a novel method of clarifying motoric subtypes. Neurocase 13(4):272–277
Li Y, Berkowitz L, Noskin G, Mehrotra S (2014) Detection of patient’s bed statuses in 3d using a microsoft kinect. In: Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, IEEE, pp 5900–5903
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814
Plagemann C, Ganapathi V, Koller D, Thrun S (2010) Real-time identification and localization of body parts from depth images. In: Robotics and Automation (ICRA), 2010 IEEE International Conference on, IEEE, pp 3108–3113
Rocha J (2012) Skeltrack. https://github.com/joaquimrocha/Skeltrack
Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124
Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45(4):427–437
Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Yu MC, Wu H, Liou JL, Lee MS, Hung YP (2012) Multiparameter sleep monitoring using a depth camera. In: International Joint Conference on Biomedical Engineering Systems and Technologies, Springer, pp 311–325
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zavala-Mondragon, L.A., Lamichhane, B., Zhang, L. et al. CNN-SkelPose: a CNN-based skeleton estimation algorithm for clinical applications. J Ambient Intell Human Comput 11, 2369–2380 (2020). https://doi.org/10.1007/s12652-019-01259-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-019-01259-5