Abstract
Automatic monitoring of exercise correctness during home physical rehabilitation could significantly increase the impact of rehabilitation treatments. To evaluate exercise quality effectively, it is necessary to extract relevant spatio-temporal motion features and compare them to an ideal exercise pattern. We argue that the features should be personalized to the patient’s needs, as the movement abilities of each patient are specifically limited and also change over time. Towards this end, we utilize the MediaPipe Pose tool to estimate 2D and 3D coordinates of skeleton joints from a monocular video stream. The joint coordinates are then processed to extract specific spatio-temporal features that are automatically weighted for each patient. This allows for personalized similarity based on the individual’s exercise patterns while requiring minimal training data and possibly offering explainable evaluations. The proposed approach is tested on the REHAB24-6 rehabilitation dataset, reaching superior effectiveness and being about 2–3 orders of magnitude more efficient than state-of-the-art solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T., Zhang, F., Grundmann, M.: Blazepose: On-device real-time body pose tracking. arXiv preprint arXiv:2006.10204 (2020)
Debnath, B., O’Brien, M., Yamaguchi, M., Behera, A.: A review of computer vision-based approaches for physical rehabilitation and assessment. Multimedia Syst. 28, 209–239 (2022). https://doi.org/10.1007/s00530-021-00815-4
Dubey, S., Dixit, M.: A comprehensive survey on human pose estimation approaches. Multimedia Syst. 1–29 (2022).https://doi.org/10.1007/s00530-022-00980-0
Exer Labs Inc: Motion engine (2022). https://patents.google.com/patent/US20220327714A1
Gimigliano, F., Negrini, S., et al.: The World Health Organization: rehabilitation 2030: a call for action. Eur. J. Phys. Rehabil. Med. 53(2), 155–168 (2017)
Google LLC: Physical training assistant system (2015). https://patents.google.com/patent/US9154739B1
He, T., Chen, Y., Wang, L., Cheng, H.: An expert-knowledge-based graph convolutional network for skeleton-based physical rehabilitation exercises assessment. IEEE Trans. Neural Syst. Rehabil. Eng. 32, 1916–1925 (2024). https://doi.org/10.1109/TNSRE.2024.3400790
Kaia Health Software GmbH: Monitoring the performance of physical exercises (2022). https://patents.google.com/patent/US11282298B2
Müller, M., Röder, T.: Motion templates for automatic classification and retrieval of motion capture data. In: ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SAC), pp. 137–146. Eurographics Association (2006)
Pereira, B., Cunha, B., Viana, P., Lopes, M., Melo, A.S.C., Sousa, A.S.P.: A machine learning app for monitoring physical therapy at home. Sensors 24(1) (2024)
Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)
Sardari, S., Sharifzadeh, S., Daneshkhah, A., Nakisa, B., Loke, S.W., Palade, V., Duncan, M.J.: Artificial intelligence for skeleton-based physical rehabilitation action evaluation: a systematic review. Comput. Biol. Med. 158 (2023).https://doi.org/10.1016/j.compbiomed.2023.106835
Sedmidubsky, J., Elias, P., Budikova, P., Zezula, P.: Content-based management of human motion data: Survey and challenges. IEEE Access 9, 64241–64255 (2021). https://doi.org/10.1109/ACCESS.2021.3075766
Senin, P.: Dynamic time warping algorithm review. Tech. Rep. Univ. Hawaii 855(1–23), 40 (2008)
Silva, D.F., Giusti, R., Keogh, E., Batista, G.E.: Speeding up similarity search under dynamic time warping by pruning unpromising alignments. Data Min. Knowl. Disc. 32, 988–1016 (2018)
Valcik, J., Sedmidubsky, J., Zezula, P.: Assessing similarity models for human-motion retrieval applications. Comput. Animation Virtual Worlds 27(5), 484–500 (2016)
Zhao, W., Reinthal, M.A., Espy, D.D., Luo, X.: Rule-based human motion tracking for rehabilitation exercises: realtime assessment, feedback, and guidance. IEEE Access 5, 21382–21394 (2017). https://doi.org/10.1109/ACCESS.2017.2759801
Acknowledgments
This work is co-financed from the state budget by the Technology Agency of the Czech Republic under the TREND Programme; project “VisioTherapy: Supporting physiotherapy treatments using computer-based movement analysis” (No. FW09020055).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jánošová, M., Budikova, P., Sedmidubsky, J. (2025). Personalized Similarity Models for Evaluating Rehabilitation Exercises from Monocular Videos. In: Chávez, E., Kimia, B., Lokoč, J., Patella, M., Sedmidubsky, J. (eds) Similarity Search and Applications. SISAP 2024. Lecture Notes in Computer Science, vol 15268. Springer, Cham. https://doi.org/10.1007/978-3-031-75823-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-75823-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-75822-5
Online ISBN: 978-3-031-75823-2
eBook Packages: Computer ScienceComputer Science (R0)