Skip to main content

Advertisement

Springer Nature Link
Log in

Virtual human pose estimation in a fire education system for children with autism spectrum disorders

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Children with autism face challenges in areas like language and social skills, which hinder their ability to undergo regular fire training. Fire is one of the most common and dangerous disaster in real life, making it essential to provide children with appropriate prevention and response education. Virtual humans can offer diverse presentation forms and interact with children with autism, thus better stimulating their willingness to participate. To train the fire safety skills of children with autism, this paper proposes the application of highly realistic virtual humans in a fire education system, aiming to improve their fire safety skills. The results show that this approach effectively enhances the fire safety skills of children with autism. To enhance the realism of virtual humans in the fire education system, this paper improves the 3D pose estimation method and proposes a multi-physical factor pose estimation algorithm. By evaluating the Mean Penetration Error (MPE) and the Percentage Not Penetrated (PNP) it was shown that the pose estimation algorithm achieved higher accuracy with only 6.4% foot penetration. We counted the number of movements and the number of movements captured by the system for all participants in the fire training and showed that the system’s motion capture accuracy was over 90%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data that support the findings of this study are not openly available due to reasons of sensitivity and are available from the corresponding author upon reasonable request.

Code availability

The codes used during the study are available from the First author by request.

References

  1. National Fire and Rescue Administration. National police and fire situation in 2022. Figshare (2023). https://www.119.gov.cn/qmxfxw/xfyw/2023/36210.shtml

  2. Senthilkumaran, M., Nazari, G., MacDermid, J.C., et al.: Effectiveness of home fire safety interventions. A systematic review and meta-analysis. PLoS One 14(5), e0215724 (2019)

    Article  Google Scholar 

  3. Miller, L.E., Dai, Y.G., Fein, D.A., et al.: Characteristics of toddlers with early versus later diagnosis of autism spectrum disorder. Autism 25(2), 416–428 (2021)

    Article  Google Scholar 

  4. Maksimović, S., Marisavljević, M., Stanojević, N., et al.: Importance of early intervention in reducing autistic symptoms and speech-language deficits in children with autism spectrum disorder. Children (Basel) 10(1), 122 (2023)

    Google Scholar 

  5. Morélot, S., Garrigou, A., Dedieu, J., et al.: Virtual reality for fire safety training: influence of immersion and sense of presence on conceptual and procedural acquisition. Comput. Educ. 166, 104145 (2021)

    Article  Google Scholar 

  6. Luo, Y.X., Li, Q., Jiang, L.R., et al.: Analysis of Chinese fire statistics during the period 1997–2017. Fire Saf. J. 125, 103400 (2021)

    Article  Google Scholar 

  7. Ke, F., Moon, J., Sokolikj, Z.: Designing and deploying a virtual social sandbox for autistic children. In: Disability and Rehabilitation: Assistive Technology, pp. 1–32 (2022)

  8. Zaini, N.A., Noor, S.F.M., Wook, T.S.M.T.: The model of game-based learning in fire safety for preschool children. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 10(9), 167–175 (2019)

    Google Scholar 

  9. García, A.S., Fernández-Sotos, P., González, P., et al.: Behavioral intention of mental health practitioners toward the adoption of virtual humans in affect recognition training. Front. Psychol. 13, 1–11 (2022)

    Article  Google Scholar 

  10. Zhao, J., Zhang, X., Lu, Y., et al.: Virtual reality technology enhances the cognitive and social communication of children with autism spectrum disorder. Front. Public Health 10, 1029392 (2022)

    Article  Google Scholar 

  11. Shimada, S., Golyanik, V., Xu, W., et al.: Physcap: physically plausible monocular 3d motion capture in real time. ACM Trans. Graph. 39(6), 1–16 (2020)

    Article  Google Scholar 

  12. Shimada, S., Golyanik, V., Xu, W., et al.: Neural monocular 3d human motion capture with physical awareness. ACM Trans. Graph. 40(4), 1–15 (2021)

    Article  Google Scholar 

  13. Grynszpan, O., Bouteiller, J., Grynszpan, S., et al.: Social gaze training for autism spectrum disorder using eye-tracking and virtual humans. Interact. Stud. 23(1), 89–115 (2022)

    Article  Google Scholar 

  14. Parsons, S., Mitchell, P.: The potential of virtual reality in social skills training for people with autistic spectrum disorders. J. Intellect. Disabil. Res. 46(5), 430–443 (2002)

    Article  Google Scholar 

  15. Ma, T., Sharifi, H., Chattopadhyay, D.: Virtual humans in health-related interventions: a meta-analysis. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–6 (2019)

  16. Consorti, F., Mancuso, R., Nocioni, M., et al.: Efficacy of virtual patients in medical education: a meta-analysis of randomized studies. Comput. Educ. 59(3), 1001–1008 (2012)

    Article  Google Scholar 

  17. Preim, B., Saalfeld, P.: A survey of virtual human anatomy education systems. Comput. Graph. 71, 132–153 (2018)

    Article  Google Scholar 

  18. Fears, N.E., Templin, T.N., Sherrod, G.M., et al.: Autistic children use less efficient goal-directed whole body movementscompared to neurotypical development. J. Autism Dev. Disord. 53(7), 1–12 (2022)

    Google Scholar 

  19. Wen, J., Gheisari, M.: iVisit-communicate for AEC education: Using virtual humans to practice communication skills in 360-degree virtual field trips. J. Comput. Civ. Eng. 37(3), 04023008 (2023)

    Article  Google Scholar 

  20. Maddalon, L., Minissi, M.E., Torres, S.C., et al.: Virtual humans for ASD intervention: a brief scoping review. MEDICINA(Buenos Aires) 83, 48–52 (2023)

    Google Scholar 

  21. Kyrlitsias, C., Michael-Grigoriou, D.: Social interaction with agents and avatars in immersive virtual environments: a survey. Front. Virtual Real. 2, 786665 (2022)

    Article  Google Scholar 

  22. Wang, S.S., Qi, Y.Q.: A virtual human’s driving method based on motion capture data. Adv. Mater. Res. 711, 500–505 (2013)

    Article  Google Scholar 

  23. He, X., Jin, X., Zheng, J.: Research on virtual human motion control based on computer-assisted multimedia simulation. Comput. Intell. Neurosci. 2022, 1902207 (2022). https://doi.org/10.1155/2022/1902207 (PMID: 35498176; PMCID: PMC9045972)

    Google Scholar 

  24. Qin, W., Tao, R., Sun, L., et al.: Muscle-driven virtual human motion generation approach based on deep reinforcement learning. Comput. Animat. Virtual Worlds 33, 3–4 (2022)

    Article  Google Scholar 

  25. Wang, J., Chai, X., Han, G., et al.: Generating 3d virtual human animation based on facial expression and human posture captured by dual cameras. Adv. Multimed. (2022)

  26. Kuramoto, A., Mizukoshi, K., Nakashima, M.: Monocular camera-based 3d human body pose estimation by generative adversarial network considering joint range of motion represented by quaternion. J. Biomech. Sci. Eng. 18(2), 22–00305 (2023)

    Article  Google Scholar 

  27. Ren, Y., Huang, D., Wang, W., et al.: BSMD: a blockchain-based secure storage mechanism for big spatio-temporal data. Future Gener. Comput. Syst. 138, 328–338 (2023)

    Article  Google Scholar 

  28. Ren, Y., Leng, Y., Cheng, Y., et al.: Secure data storage based on blockchain and coding in edge computing. Math. Biosci. Eng. 16(4), 1874–1892 (2019)

    Article  MathSciNet  Google Scholar 

  29. Adhvaryu, K.P., Karthikbabu, S., Rao, P.T.: Motor performance of children with attention deficit hyperactivity disorder: focus on the Bruininks–Oseretsky test of motor proficiency. Clin. Exp. Pediatrics 65(11), 512 (2022)

    Article  Google Scholar 

  30. Kushwaha, M., Choudhary, J., Singh, D.P.: Enhancement of human 3d pose estimation using a novel concept of depth prediction with pose alignment from a single 2d image. Comput. Graph. 107, 172–185 (2022)

    Article  Google Scholar 

  31. Habermann, M., Xu, W., Zollhofer, M., et al.: Deepcap: Monocular human performance capture using weak supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5052–5063 (2020)

  32. Ren, Y., Leng, Y., Qi, J., et al.: Multiple cloud storage mechanism based on blockchain in smart homes. Future Gener. Comput. Syst. 115, 304–313 (2021)

    Article  Google Scholar 

  33. Hong, K.H., Kim, D.Y., Kang, H.B., et al.: A preliminary study on motor ability of preschool aged children by using Bruininks–Oseretsky test of motor proficiency-2 (bot-2) short form. J. Korean Acad. Sens. Integr. 14(1), 31–40 (2016)

    Google Scholar 

  34. Jírovec, J., Musálek, M., Mess, F.: Test of motor proficiency second edition (bot-2): compatibility of the complete and short form and its usefulness for middle-age school children. Front. Pediatrics 7, 153 (2019)

Download references

Acknowledgements

The authors thank all the subjects who participated in this study. The authors would like to thank the anonymous reviewers whose comments/suggestions helped improve and clarify the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China (No. 62072249).

Author information

Author notes

  1. Yangyang Guo and Hongye Liu have contributed equally to this work and share the first authorship.

Authors and Affiliations

  1. School of Computer Science, Nanjing University of Information Science & Technology, Nanjing, 210044, Jiangsu, China

    Yangyang Guo & Yongjun Ren

  2. School of Acupuncture-Moxibustion and Tuina, Nanjing University of Chinese Medicine, Nanjing, 210023, Jiangsu, China

    Hongye Liu

  3. School of Health Preservation and Rehabilitation, Nanjing University of Chinese Medicine, Nanjing, 210023, Jiangsu, China

    Hongye Liu

  4. Department of Rehabilitation Medicine, Children’s Hospital of Nanjing Medical University, Nanjing, 210093, Jiangsu, China

    Yaojin Sun

Authors
  1. Yangyang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongye Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaojin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongjun Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y.G.: conceptualization, methodology, software, formal analysis, investigation, writing—original draft. H.L.: conceptualization, validation, investigation, data curation, writing—original draft. Y.S.: data curation, writing—review and editing, validation, investigation, funding acquisition. Y.R.:writing—review and editing, funding acquisition, supervision.

Corresponding author

Correspondence to Yongjun Ren.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical approval

Approval to conduct this study was granted by the institutional ethics committee at the hospital (IRB No. 202111123-1). Participants’ legal guardians who were parents of infants in this study received written information explaining the aims, processes, risks, and benefits of the study, and informed consent was obtained for all observations and reconfirmed throughout the data collection period.

Consent to participate

Informed consent was obtained from legal guardians.

Consent for publication

Additional informed consent was obtained from all individual participants for whom identifying information is included in this article.

Additional information

Communicated by P. Pala.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Y., Liu, H., Sun, Y. et al. Virtual human pose estimation in a fire education system for children with autism spectrum disorders. Multimedia Systems 30, 84 (2024). https://doi.org/10.1007/s00530-024-01274-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01274-3

Keywords

Search

Navigation