Abstract
While human motion analysis has been widely addressed in recent years, the specific task of rehabilitation motion assessment remains challenging due to the lack of available annotated data. To overcome this challenge, data augmentation can be considered. However, classical augmentation techniques applied to human motion sequences often result in meaningless movements. Moreover, in rehabilitation assessment, labels are often continuous values illustrating the quality of a movement. Hence, associating a continuous label to augmented data is not straightforward. In this work, we propose to address data augmentation using an averaging method, called shapeDBA, adapted to rehabilitation motion sequences represented as multivariate time series. We extend the original proposal by weighting the average, hence allowing us to infer continuous labels associated to augmented motion sequences. We evaluated our proposed method on the Kimore dataset. Experimental results show that our method generates coherent rehabilitation sequences that can be efficiently used to extend a small dataset for rehabilitation assessment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blanchard, A., Nguyen, S.M., Devanne, M., Simonnet, M., Le Goff-Pronost, M., Rémy-Néris, O.: Technical feasibility of supervision of stretching exercises by a humanoid robot coach for chronic low back pain: the r-cool randomized trial. BioMed Res. Int. 2022(1), 1–10 (2022). https://doi.org/10.1155/2022/5667223
Capecci, M., et al.: A hidden semi-Markov model based approach for rehabilitation exercise assessment. J. Biomed. Inform. 78, 1–11 (2018)
Capecci, M., et al.: The kimore dataset: kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 27(7), 1436–1448 (2019)
Chen, J., Yang, W., Liu, C., Yao, L.: A data augmentation method for skeleton-based action recognition with relative features. Appl. Sci. 11(23), 11481 (2021)
Deb, S., Islam, M.F., Rahman, S., Rahman, S.: Graph convolutional networks for assessment of physical rehabilitation exercises. IEEE Trans. Neural Syst. Rehabil. Eng. 30, 410–419 (2022)
Devanne, M., et al.: Multi-level motion analysis for physical exercises assessment in kinaesthetic rehabilitation. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 529–534. IEEE (2017)
Forestier, G., Petitjean, F., Webb, G., Dau, H.A., Keogh, E.: Generating synthetic time series to augment sparse datasets. In: IEEE International Conference on Data Mining (ICDM), pp. 865–870 (2017). https://doi.org/10.1109/ICDM.2017.106
Guijo-Rubio, D., Middlehurst, M., Arcencio, G., Silva, D.F., Bagnall, A.: Unsupervised feature based algorithms for time series extrinsic regression. Data Min. Knowl. Discov. 1–45 (2024)
Guo, C., et al.: Action2motion: conditioned generation of 3d human motions. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2021–2029 (2020)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Huynh-The, T., Hua, C.H., Kim, D.S.: Encoding pose features to images with data augmentation for 3-D action recognition. IEEE Trans. Industr. Inf. 16(5), 3100–3111 (2019)
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Lite: light inception with boosting techniques for time series classification. In: 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA), pp. 1–10. IEEE (2023)
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Establishing a unified evaluation framework for human motion generation: a comparative analysis of metrics. arXiv preprint arXiv:2405.07680 (2024)
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: A supervised variational auto-encoder for human motion generation using convolutional neural networks. In: 4th International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI) (2024)
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G.: Weighted elastic barycetner averaging to augment time series data (2024). https://github.com/MSD-IRIMAS/Augmenting-TSC-Elastic-Averaging
Ismail-Fawaz, A., Devanne, M., Weber, J., Forestier, G.: Deep learning for time series classification using new hand-crafted convolution filters. In: 2022 IEEE International Conference on Big Data (Big Data), pp. 972–981. IEEE (2022)
Ismail-Fawaz, A., et al.: Shapedba: generating effective time series prototypes using shapedtw barycenter averaging. In: International Workshop on Advanced Analytics and Learning on Temporal Data, pp. 127–142. Springer (2023)
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Data augmentation using synthetic data for time series classification with deep residual networks. In: ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data (2018)
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Deep learning for time series classification: a review. Data Min. Knowl. Disc. 33(4), 917–963 (2019)
Ismayilzada, E., Devanne, M., , Weber, J., Forestier, G.: Time series extrinsic regression for physical rehabilitation assessment. In: Upper Rhine Artificial Intelligence Symposium (URAI) (2023). undefined
Lee, I., Kim, D., Kang, S., Lee, S.: Ensemble deep learning for skeleton-based action recognition using temporal sliding LSTM networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1012–1020 (2017)
Liao, Y., Vakanski, A., Xian, M.: A deep learning framework for assessing physical rehabilitation exercises. IEEE Trans. Neural Syst. Rehabil. Eng. 28(2), 468–477 (2020)
Middlehurst, M., et al.: aeon: a python toolkit for learning from time series. arXiv preprint arXiv:2406.14231 (2024)
Mourchid, Y., Slama, R.: MR-STGN: Multi-residual Spatio temporal graph network using attention fusion for patient action assessment. In: 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE (2023)
Müller, M.: Dynamic time warping. Inf. Retrieval Music Motion, 69–84 (2007)
Naeem, M.F., Oh, S.J., Uh, Y., Choi, Y., Yoo, J.: Reliable fidelity and diversity metrics for generative models. In: International Conference on Machine Learning, pp. 7176–7185. PMLR (2020)
Nguyen, S., Devanne, M., Remy Neris, O., Lempereur, M., Thepaut, A.: A medical low-back pain physical rehabilitation database for human body movement analysis. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2024)
Petitjean, F., Ketterlin, A., Gançarski, P.: A global averaging method for dynamic time warping, with applications to clustering. Pattern Recogn. 44(3), 678–693 (2011)
Petrovich, M., Black, M.J., Varol, G.: Action-conditioned 3d human motion synthesis with transformer VAE. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10985–10995 (2021)
Pialla, G., Devanne, M., Weber, J., Idoumghar, L., Forestier, G.: Data augmentation for time series classification with deep learning models. In: International Workshop on Advanced Analytics and Learning on Temporal Data (2022)
Rao, H., Xu, S., Hu, X., Cheng, J., Hu, B.: Augmented skeleton based contrastive action learning with momentum LSTM for unsupervised action recognition. Inf. Sci. 569, 90–109 (2021)
Vakanski, A., Jun, H.P., Paul, D., Baker, R.: A data set of human body movements for physical rehabilitation exercises. Data 3(1), 2 (2018)
Virtanen, P., et al.: Scipy 1.0: fundamental algorithms for scientific computing in python. Nat. Methods 17(3), 261–272 (2020)
Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578–1585. IEEE (2017)
Xin, C., Kim, S., Cho, Y., Park, K.S.: Enhancing human action recognition with 3D skeleton data: a comprehensive study of deep learning and data augmentation. Electronics 13(4), 747 (2024)
Zhao, J., Itti, L.: shapeDTW: shape dynamic time warping. Pattern Recogn. 74, 171–184 (2018)
Acknowledgment
This work was supported by the ANR DELEGATION project (grant ANR-21-CE23-0014) of the French Agence Nationale de la Recherche. The authors would like to acknowledge the High Performance Computing Center of the University of Strasbourg for supporting this work by providing scientific support and access to computing resources. Part of the computing resources were funded by the Equipex Equip@Meso project (Programme Investissements d’Avenir) and the CPER Alsacalcul/Big Data. The authors would also like to thank the creators and providers of the Kimore dataset.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., Forestier, G. (2025). Weighted Average of Human Motion Sequences for Improving Rehabilitation Assessment. In: Lemaire, V., et al. Advanced Analytics and Learning on Temporal Data. AALTD 2024. Lecture Notes in Computer Science(), vol 15433. Springer, Cham. https://doi.org/10.1007/978-3-031-77066-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-77066-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-77065-4
Online ISBN: 978-3-031-77066-1
eBook Packages: Computer ScienceComputer Science (R0)