Skip to main content
Springer Nature Link
Log in

Patient 3D body pose estimation from pressure imaging

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

In-bed motion monitoring has become of great interest for a variety of clinical applications. Image-based approaches could be seen as a natural non-intrusive approach for this purpose; however, video devices require special challenging settings for a clinical environment. We propose to estimate the patient’s posture from pressure sensors’ data mapped to images.

Methods

We introduce a deep learning method to retrieve human poses from pressure sensors data. In addition, we present a second approach that is based on a hashing content-retrieval approach.

Results

Our results show good performance with both presented methods even in poses where the subject has minimal contact with the sensors. Moreover, we show that deep learning approaches could be used in this medical application despite the limited amount of available training data. Our ConvNet approach provides an overall posture even when the patient has less contact with the mattress surface. In addition, we show that both methods could be used in real-time patient monitoring.

Conclusions

We have provided two methods to successfully perform real-time in-bed patient pose estimation, which is robust to different sizes of patient and activities. Furthermore, it can provide an overall posture even when the patient has less contact with the mattress surface.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Achilles F, Ichim AE, Coskun H, Tombari F, Noachtar S, Navab N (2016) Patient MoCap: human pose estimation under blanket occlusion for hospital monitoring applications. In: MICCAI (1), Lecture Notes in Computer Science, vol 9900, pp 491–499

  2. Alaziz M, Jia Z, Liu J, Howard R, Chen Y, Zhang Y (2016) Motion scale: a body motion monitoring system using bed-mounted wireless load cells. In: 2016 IEEE first international conference on connected health: applications, systems and engineering technologies (CHASE), pp 183–192, https://doi.org/10.1109/CHASE.2016.13

  3. Auvinet E, Multon F, Saint-Arnaud A, Rousseau J, Meunier J (2011) Fall detection with multiple cameras: an occlusion-resistant method based on 3-d silhouette vertical distribution. IEEE Trans Inf Technol Biomed 15(2):290–300

    Article  PubMed  Google Scholar 

  4. Banerjee T, Enayati M, Keller JM, Skubic M, Popescu M, Rantz M (2014) Monitoring patients in hospital beds using unobtrusive depth sensors. In: 2014 36th annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE, pp 5904–5907

  5. Belagiannis V, Zisserman A (2017) Recurrent human pose estimation. In: 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), IEEE, pp 468–475

  6. Belagiannis V, Rupprecht C, Carneiro G, Navab N (2015) Robust optimization for deep regression. In: 2015 IEEE international conference on computer vision (ICCV), pp 2830–2838

  7. Belagiannis V, Wang X, Shitrit HB, Hashimoto K, Stauder R, Aoki Y, Kranzfelder M, Schneider A, Fua P, Ilic S, Feussner H, Navab N (2016) Parsing human skeletons in an operating room. Mach Vis Appl 27(7):1035–1046. https://doi.org/10.1007/s00138-016-0792-4

    Article  Google Scholar 

  8. Casas L, Mürwald C, Achilles F, Mateus D, Huber D, Navab N, Demirci S (2018) Human pose estimation from pressure sensor data. In: Maier A, Deserno TM, Handels H, Maier-Hein KH, Palm C, Tolxdorff T (eds) Bildverarbeitung für die Medizin 2018. Springer, Heidelberg, pp 285–290

    Chapter  Google Scholar 

  9. Chu X, Yang W, Ouyang W, Ma C, Yuille AL, Wang X (2017) Multi-context attention for human pose estimation, Vol 1, no. 2. arXiv preprint arXiv:1702.07432

  10. Gong Y, Lazebnik S, Gordo A, Perronnin F (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929. https://doi.org/10.1109/TPAMI.2012.193

    Article  PubMed  Google Scholar 

  11. Grimm R, Bauer S, Sukkau J, Hornegger J, Greiner G (2012) Markerless estimation of patient orientation, posture and pose using range and pressure imaging. Int J Comput Assist Radiol Surg 7(6):921–929. https://doi.org/10.1007/s11548-012-0694-5

    Article  PubMed  Google Scholar 

  12. Harada T, Sato T, Mori T (2001) Pressure distribution image based human motion tracking system using skeleton and surface integration model. In: Proceedings 2001 ICRA. IEEE international conference on robotics and automation (Cat. No. 01CH37164), vol 4, pp 3201–3207. https://doi.org/10.1109/ROBOT.2001.933111

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  14. Heydarzadeh M, Nourani M, Ostadabbas S (2016) In-bed posture classification using deep autoencoders. In: 2016 IEEE 38th annual international conference of the engineering in medicine and biology society (EMBC), IEEE, pp 3839–3842. https://doi.org/10.1109/EMBC.2016.7591565

  15. Hoque E, Dickerson RF, Stankovic JA (2010) Monitoring body positions and movements during sleep using wisps. In: Wireless health 2010, ACM, New York, NY, USA, WH ’10, pp 44–53. https://doi.org/10.1145/1921081.1921088

  16. Kadkhodamohammadi A, Gangi A, de Mathelin M, Padoy N (2017) A multi-view RGB-D approach for human pose estimation in operating rooms. In: 2017 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 363–372

  17. Kehl W, Tombari F, Navab N, Ilic S, Lepetit V (2016) Hashmod: a hashing method for scalable 3D object detection. arXiv preprint arXiv:1607.06062

  18. Kortelainen JM, Van Gils M, Pärkkä J (2012) Multichannel bed pressure sensor for sleep monitoring. In: Computing in cardiology (CinC), 2012, IEEE, pp 313–316

  19. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  20. Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: 2009 IEEE 12th international conference on computer vision, IEEE, pp 2130–2137

  21. Newell A, Yang K, Deng J (2016) Stacked hourglass networks for human pose estimation. In: European conference on computer vision, Springer, pp 483–499

  22. Papandreou G, Zhu T, Chen LC, Gidaris S, Tompson J, Murphy K (2018) PersonLab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. arXiv preprint arXiv:1803.08225

  23. Papazov C, Burschka D (2010) An efficient ransac for 3d object recognition in noisy and occluded scenes. In: Asian conference on computer vision, Springer, pp 135–148

  24. Placht S, Stancanello J, Schaller C, Balda M, Angelopoulou E (2012) Fast time-of-flight camera based surface registration for radiotherapy patient positioning. Med Phys 39(1):4–17

    Article  PubMed  Google Scholar 

  25. Shakhnarovich G, Viola P, Darrell T (2003) Fast pose estimation with parameter-sensitive hashing. In: Null, IEEE, p 750

  26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  27. Toms M (2000) Patient movement detection. https://www.google.com/patents/US6036660. US Patent 6,036,660

  28. Vedaldi A, Lenc K (2015) Matconvnet—convolutional neural networks for matlab. In: Proceeding of the ACM international conference on multimedia

  29. Wei SE, Ramakrishna V, Kanade T, Sheikh Y (2016) Convolutional pose machines. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4724–4732

  30. Yu X, Zhang S, Liu B, Zhong L, Metaxas DN (2013) Large scale medical image search via unsupervised PCA hashing. In: 2013 IEEE conference on computer vision and pattern recognition workshops (CVPRW), IEEE, pp 393–398

  31. Zhang X, Liu W, Dundar M, Badve S, Zhang S (2015) Towards large-scale histopathological image analysis: hashing-based image retrieval. IEEE Trans Med Imag 34(2):496–506

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Aided Medical Procedures, Technische Universität München, Boltzmannstr 3, 85748, Garching, Germany

    Leslie Casas, Nassir Navab & Stefanie Demirci

Authors
  1. Leslie Casas
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Nassir Navab
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Stefanie Demirci
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Leslie Casas.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Casas, L., Navab, N. & Demirci, S. Patient 3D body pose estimation from pressure imaging. Int J CARS 14, 517–524 (2019). https://doi.org/10.1007/s11548-018-1895-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1895-3

Keywords