Skip to main content
SpringerLink
Log in

Vision-Based Human Posture Detection from a Virtual Home-Care Unmanned Aerial Vehicle

  • Conference paper
  • First Online:
Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence (IWINAC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13259))

  • 1212 Accesses

Abstract

Monitoring is essential to provide assistance to people who require home care due to their age or health condition. This paper presents the vision-based detection of three postures of a person (standing, sitting and laying down) from an unmanned aerial vehicle. The proposal uses the MediaPipe Pose Python module, considering only seven skeleton points and a set of trigonometric calculations. The work is evaluated in a Unity virtual reality (VR) environment that simulates the monitoring process of an assistant UAV. The images acquired by the UAV’s on-board camera are sent from the VR visualiser to the Python module via the Message Queue Telemetry Transport (MQTT) protocol. The simulation shows very promising results for the detection of a person’s postures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Belagiannis, V., Zisserman, A.: Recurrent human pose estimation. In: 2017 12th IEEE International Conference on Automatic Face Gesture Recognition, FG 2017, pp. 468–475 (2017). https://doi.org/10.1109/FG.2017.64

  2. Belmonte, L.M., García, A.S., Morales, R., de la Vara, J.L., López de la Rosa, F., Fernández-Caballero, A.: Feeling of safety and comfort towards a socially assistive unmanned aerial vehicle that monitors people in a virtual home. Sensors 21(3) (2021). https://doi.org/10.3390/s21030908

  3. Belmonte, L.M., García, A.S., Segura, E., Novais, P., Morales, R., Fernández-Caballero, A.: Virtual reality simulation of a quadrotor to monitor dependent people at home. IEEE Trans. Emerg. Top. Comput. 9(3), 1301–1315 (2021). https://doi.org/10.1109/TETC.2020.3000352

    Article  Google Scholar 

  4. Belmonte, L.M., Morales, R., García, A.S., Segura, E., Novais, P., Fernández-Caballero, A.: Assisting dependent people at home through autonomous unmanned aerial vehicles. In: Novais, P., Lloret, J., Chamoso, P., Carneiro, D., Navarro, E., Omatu, S. (eds.) ISAmI 2019. AISC, vol. 1006, pp. 216–223. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-24097-4_26

    Chapter  Google Scholar 

  5. Belmonte, L.M., Morales, R., García, A.S., Segura, E., Novais, P., Fernández-Caballero, A.: Trajectory planning of a quadrotor to monitor dependent people. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds.) IWINAC 2019. LNCS, vol. 11486, pp. 212–221. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19591-5_22

    Chapter  Google Scholar 

  6. Belmonte, L.M., Morales, R., Fernández-Caballero, A.: Computer vision in autonomous unmanned aerial vehicles - a systematic mapping study. Appl. Sci. 9(15) (2019). https://doi.org/10.3390/app9153196

  7. Carretero, S., Garcés, J., Ródenas, F.: Evaluation of the home help service and its impact on the informal caregiver’s burden of dependent elders. Int. J. Geriatr. Psychiatry 22(8), 738–749 (2007). https://doi.org/10.1002/gps.1733

    Article  Google Scholar 

  8. Carretero, S., Garcés, J., Ródenas, F., Sanjosé, V.: The informal caregiver’s burden of dependent people: theory and empirical review. Arch. Gerontol. Geriatr. 49(1), 74–79 (2009). https://doi.org/10.1016/j.archger.2008.05.004

    Article  Google Scholar 

  9. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 428–441. Springer, Heidelberg (2006). https://doi.org/10.1007/11744047_33

    Chapter  Google Scholar 

  10. Halder, A., Tayade, A.: Real-time vernacular sign language recognition using MediaPipe and machine learning. Int. J. Res. Publ. Rev. 2, 9–17 (2021)

    Google Scholar 

  11. Lugaresi, C., et al.: MediaPipe: a framework for building perception pipelines (2019)

    Google Scholar 

  12. Martínez, A., Belmonte, L.M., García, A.S., Fernández-Caballero, A., Morales, R.: Facial emotion recognition from an unmanned flying social robot for home care of dependent people. Electronics 10(7) (2021). https://doi.org/10.3390/electronics10070868

  13. Moon, S., Park, Y., Ko, D.W., Suh, I.H.: Multiple kinect sensor fusion for human skeleton tracking using Kalman filtering. Int. J. Adv. Rob. Syst. 13(2), 65 (2016). https://doi.org/10.5772/62415

    Article  Google Scholar 

  14. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  15. Penmetsa, S., Minhuj, F., Singh, A., Omkar, S.: Autonomous UAV for suspicious action detection using pictorial human pose estimation and classification. ELCVIA Electron. Lett. Comput. Vis. Image Anal. 13(1), 18 (2014). https://doi.org/10.5565/rev/elcvia.582

    Article  Google Scholar 

  16. Pietraszek, T.: Using adaptive alert classification to reduce false positives in intrusion detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 102–124. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30143-1_6

    Chapter  Google Scholar 

  17. Puwein, J., Ballan, L., Ziegler, R., Pollefeys, M.: Joint camera pose estimation and 3d human pose estimation in a multi-camera setup. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 473–487. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16808-1_32

    Chapter  Google Scholar 

  18. Rudol, P., Doherty, P.: Human body detection and geolocalization for UAV search and rescue missions using color and thermal imagery. In: 2008 IEEE Aerospace Conference, pp. 1–8. IEEE (2008)

    Google Scholar 

  19. Sanchez-Comas, A., Synnes, K., Hallberg, J.: Hardware for recognition of human activities: a review of smart home and AAL related technologies. Sensors 20(15), 4227 (2020). https://doi.org/10.3390/s20154227

    Article  Google Scholar 

  20. Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N.: Human skeleton tracking from depth data using geodesic distances and optical flow. Image Vis. Comput. 30(3), 217–226 (2012). Best of Automatic Face and Gesture Recognition 2011. https://doi.org/10.1016/j.imavis.2011.12.001

  21. Seint, P., Zin, T., Tin, P.: Intelligent monitoring for elder care using vision-based technology. Int. J. Innov. Comput. Inf. Control 17(3), 905–918 (2021). https://doi.org/10.24507/ijicic.17.03.905

  22. Watkins, L., Fairbanks, K.D., Li, C., Yang, M., Robinson, W.H., Rubin, A.: A black box approach to inferring, characterizing, and breaking native device tracking autonomy. In: 2020 11th IEEE Annual Ubiquitous Computing, Electronics Mobile Communication Conference (UEMCON), pp. 0303–0308 (2020). https://doi.org/10.1109/UEMCON51285.2020.9298163

  23. Xu, H., Bazavan, E.G., Zanfir, A., Freeman, W.T., Sukthankar, R., Sminchisescu, C.: GHUM & GHUML: generative 3d human shape and articulated pose models. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6183–6192 (2020). https://doi.org/10.1109/CVPR42600.2020.00622

Download references

Acknowledgements

Grants PID2020-115220RB-C21 and EQC2019-006063-P funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way to make Europe”. This work was also partially supported by CIBERSAM of the Instituto de Salud Carlos III. This work has also been partially supported by Portuguese Fundação para a Ciência e a Tecnologia - FCT, I.P. under the project UIDB/04524/2020 and by Portuguese National funds through FITEC - Programa Interface, with reference CIT “INOV - INESC Inovação - Financiamento Base“. This work has also been partially supported by Junta de Comunidades de Castilla-La Mancha/ESF (grant No. SBPLY/21/180501/000030).

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática de Albacete, Unidad Multidisciplinar en Neurocognición y Emoción, 02071, Albacete, Spain

    Andrés Bustamante, Lidia M. Belmonte, Pascual González, Antonio Fernández-Caballero & Rafael Morales

  2. Universidad de Castilla-La Mancha, E.T.S. Ingenieros Industriales de Albacete, 02071, Albacete, Spain

    Lidia M. Belmonte, Antonio Fernández-Caballero & Rafael Morales

  3. Polytechnic Institute of Leiria, Computer Science and Communications Research Centre, School of Technology and Management, 2411-901, Leiria, Portugal

    António Pereira

  4. INOV INESC INOVAÇÃO, Institute of New Technologies—Leiria Office, 2411-901, Leiria, Portugal

    António Pereira

  5. Universidad de Castilla-La Mancha, Escula Superior de Ingeniería Informática de Albacete, 02071, Albacete, Spain

    Pascual González

  6. Biomedical Research Networking Centre in Mental Health (CIBERSAM), 28016, Madrid, Spain

    Pascual González & Antonio Fernández-Caballero

Authors
  1. Andrés Bustamante
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lidia M. Belmonte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. António Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pascual González
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio Fernández-Caballero
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rafael Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafael Morales .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Universidad Nacional de Educación a Distancia, Madrid, Spain

    José Ramón Álvarez-Sánchez

  3. Universidad Nacional de Educación a Distancia, Madrid, Spain

    Félix de la Paz López

  4. Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bustamante, A., Belmonte, L.M., Pereira, A., González, P., Fernández-Caballero, A., Morales, R. (2022). Vision-Based Human Posture Detection from a Virtual Home-Care Unmanned Aerial Vehicle. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06527-9_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06526-2

  • Online ISBN: 978-3-031-06527-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation