Skip to main content

Weighted Average of Human Motion Sequences for Improving Rehabilitation Assessment

  • Conference paper
  • First Online:
Advanced Analytics and Learning on Temporal Data (AALTD 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 15433))

  • 104 Accesses

Abstract

While human motion analysis has been widely addressed in recent years, the specific task of rehabilitation motion assessment remains challenging due to the lack of available annotated data. To overcome this challenge, data augmentation can be considered. However, classical augmentation techniques applied to human motion sequences often result in meaningless movements. Moreover, in rehabilitation assessment, labels are often continuous values illustrating the quality of a movement. Hence, associating a continuous label to augmented data is not straightforward. In this work, we propose to address data augmentation using an averaging method, called shapeDBA, adapted to rehabilitation motion sequences represented as multivariate time series. We extend the original proposal by weighting the average, hence allowing us to infer continuous labels associated to augmented motion sequences. We evaluated our proposed method on the Kimore dataset. Experimental results show that our method generates coherent rehabilitation sequences that can be efficiently used to extend a small dataset for rehabilitation assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Blanchard, A., Nguyen, S.M., Devanne, M., Simonnet, M., Le Goff-Pronost, M., Rémy-Néris, O.: Technical feasibility of supervision of stretching exercises by a humanoid robot coach for chronic low back pain: the r-cool randomized trial. BioMed Res. Int. 2022(1), 1–10 (2022). https://doi.org/10.1155/2022/5667223

    Article  Google Scholar 

  2. Capecci, M., et al.: A hidden semi-Markov model based approach for rehabilitation exercise assessment. J. Biomed. Inform. 78, 1–11 (2018)

    Article  MATH  Google Scholar 

  3. Capecci, M., et al.: The kimore dataset: kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 27(7), 1436–1448 (2019)

    Article  MATH  Google Scholar 

  4. Chen, J., Yang, W., Liu, C., Yao, L.: A data augmentation method for skeleton-based action recognition with relative features. Appl. Sci. 11(23), 11481 (2021)

    Article  MATH  Google Scholar 

  5. Deb, S., Islam, M.F., Rahman, S., Rahman, S.: Graph convolutional networks for assessment of physical rehabilitation exercises. IEEE Trans. Neural Syst. Rehabil. Eng. 30, 410–419 (2022)

    Article  MATH  Google Scholar 

  6. Devanne, M., et al.: Multi-level motion analysis for physical exercises assessment in kinaesthetic rehabilitation. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 529–534. IEEE (2017)

    Google Scholar 

  7. Forestier, G., Petitjean, F., Webb, G., Dau, H.A., Keogh, E.: Generating synthetic time series to augment sparse datasets. In: IEEE International Conference on Data Mining (ICDM), pp. 865–870 (2017). https://doi.org/10.1109/ICDM.2017.106

  8. Guijo-Rubio, D., Middlehurst, M., Arcencio, G., Silva, D.F., Bagnall, A.: Unsupervised feature based algorithms for time series extrinsic regression. Data Min. Knowl. Discov. 1–45 (2024)

    Google Scholar 

  9. Guo, C., et al.: Action2motion: conditioned generation of 3d human motions. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2021–2029 (2020)

    Google Scholar 

  10. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  11. Huynh-The, T., Hua, C.H., Kim, D.S.: Encoding pose features to images with data augmentation for 3-D action recognition. IEEE Trans. Industr. Inf. 16(5), 3100–3111 (2019)

    Article  MATH  Google Scholar 

  12. Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Lite: light inception with boosting techniques for time series classification. In: 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–10. IEEE (2023)

    Google Scholar 

  13. Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Establishing a unified evaluation framework for human motion generation: a comparative analysis of metrics. arXiv preprint arXiv:2405.07680 (2024)

  14. Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: A supervised variational auto-encoder for human motion generation using convolutional neural networks. In: 4th International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI) (2024)

    Google Scholar 

  15. Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Weighted elastic barycetner averaging to augment time series data (2024). https://github.com/MSD-IRIMAS/Augmenting-TSC-Elastic-Averaging

  16. Ismail-Fawaz, A., Devanne, M., Weber, J., Forestier, G.: Deep learning for time series classification using new hand-crafted convolution filters. In: 2022 IEEE International Conference on Big Data (Big Data), pp. 972–981. IEEE (2022)

    Google Scholar 

  17. Ismail-Fawaz, A., et al.: Shapedba: generating effective time series prototypes using shapedtw barycenter averaging. In: International Workshop on Advanced Analytics and Learning on Temporal Data, pp. 127–142. Springer (2023)

    Google Scholar 

  18. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Data augmentation using synthetic data for time series classification with deep residual networks. In: ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data (2018)

    Google Scholar 

  19. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Deep learning for time series classification: a review. Data Min. Knowl. Disc. 33(4), 917–963 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  20. Ismayilzada, E., Devanne, M., , Weber, J., Forestier, G.: Time series extrinsic regression for physical rehabilitation assessment. In: Upper Rhine Artificial Intelligence Symposium (URAI) (2023). undefined

    Google Scholar 

  21. Lee, I., Kim, D., Kang, S., Lee, S.: Ensemble deep learning for skeleton-based action recognition using temporal sliding LSTM networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1012–1020 (2017)

    Google Scholar 

  22. Liao, Y., Vakanski, A., Xian, M.: A deep learning framework for assessing physical rehabilitation exercises. IEEE Trans. Neural Syst. Rehabil. Eng. 28(2), 468–477 (2020)

    Article  MATH  Google Scholar 

  23. Middlehurst, M., et al.: aeon: a python toolkit for learning from time series. arXiv preprint arXiv:2406.14231 (2024)

  24. Mourchid, Y., Slama, R.: MR-STGN: Multi-residual Spatio temporal graph network using attention fusion for patient action assessment. In: 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE (2023)

    Google Scholar 

  25. Müller, M.: Dynamic time warping. Inf. Retrieval Music Motion, 69–84 (2007)

    Google Scholar 

  26. Naeem, M.F., Oh, S.J., Uh, Y., Choi, Y., Yoo, J.: Reliable fidelity and diversity metrics for generative models. In: International Conference on Machine Learning, pp. 7176–7185. PMLR (2020)

    Google Scholar 

  27. Nguyen, S., Devanne, M., Remy Neris, O., Lempereur, M., Thepaut, A.: A medical low-back pain physical rehabilitation database for human body movement analysis. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2024)

    Google Scholar 

  28. Petitjean, F., Ketterlin, A., Gançarski, P.: A global averaging method for dynamic time warping, with applications to clustering. Pattern Recogn. 44(3), 678–693 (2011)

    Article  MATH  Google Scholar 

  29. Petrovich, M., Black, M.J., Varol, G.: Action-conditioned 3d human motion synthesis with transformer VAE. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10985–10995 (2021)

    Google Scholar 

  30. Pialla, G., Devanne, M., Weber, J., Idoumghar, L., Forestier, G.: Data augmentation for time series classification with deep learning models. In: International Workshop on Advanced Analytics and Learning on Temporal Data (2022)

    Google Scholar 

  31. Rao, H., Xu, S., Hu, X., Cheng, J., Hu, B.: Augmented skeleton based contrastive action learning with momentum LSTM for unsupervised action recognition. Inf. Sci. 569, 90–109 (2021)

    Article  MATH  Google Scholar 

  32. Vakanski, A., Jun, H.P., Paul, D., Baker, R.: A data set of human body movements for physical rehabilitation exercises. Data 3(1), 2 (2018)

    Article  Google Scholar 

  33. Virtanen, P., et al.: Scipy 1.0: fundamental algorithms for scientific computing in python. Nat. Methods 17(3), 261–272 (2020)

    Article  MATH  Google Scholar 

  34. Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578–1585. IEEE (2017)

    Google Scholar 

  35. Xin, C., Kim, S., Cho, Y., Park, K.S.: Enhancing human action recognition with 3D skeleton data: a comprehensive study of deep learning and data augmentation. Electronics 13(4), 747 (2024)

    Article  MATH  Google Scholar 

  36. Zhao, J., Itti, L.: shapeDTW: shape dynamic time warping. Pattern Recogn. 74, 171–184 (2018)

    Article  MATH  Google Scholar 

Download references

Acknowledgment

This work was supported by the ANR DELEGATION project (grant ANR-21-CE23-0014) of the French Agence Nationale de la Recherche. The authors would like to acknowledge the High Performance Computing Center of the University of Strasbourg for supporting this work by providing scientific support and access to computing resources. Part of the computing resources were funded by the Equipex Equip@Meso project (Programme Investissements d’Avenir) and the CPER Alsacalcul/Big Data. The authors would also like to thank the creators and providers of the Kimore dataset.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ali Ismail-Fawaz .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G. (2025). Weighted Average of Human Motion Sequences for Improving Rehabilitation Assessment. In: Lemaire, V., et al. Advanced Analytics and Learning on Temporal Data. AALTD 2024. Lecture Notes in Computer Science(), vol 15433. Springer, Cham. https://doi.org/10.1007/978-3-031-77066-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-77066-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-77065-4

  • Online ISBN: 978-3-031-77066-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics