Abstract
Human action recognition aims at extracting features on top of human skeletons and estimating human pose. It has received increasing attention in recent years. However, existing methods capture only the action information while in a real world application such as cognitive assessment, we need to measure the executive functioning that helps psychiatrists to identify some mental disease such as Alzheimer, Schizophrenia and ADHD. In this paper, we propose a skeleton-based action recognition named Mind-In-Action (MIA) for cognitive assessment. MIA integrates a pose estimator to extract the human body joints and then automatically measures the executive functioning employing the distance and elbow angle calculation. Three score functions were designed to measure the executive functioning: the accuracy score, the rhythm score and the functioning score. We evaluate our model on two different datasets and show that our approach significantly outperforms the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: Openpose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021)
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., Hu, W.: Channel-wise topology refinement graph convolution for skeleton-based action recognition. In: ICCV, pp. 13359–13368 (2021)
Cormier, E.: Attention deficit/hyperactivity disorder: a review and update. J. Pediatr. Nurs. 23(5), 345–357 (2008)
Duan, H., Zhao, Y., Chen, K., Lin, D., Dai, B.: Revisiting skeleton-based action recognition. In: CVPR (2022)
Farha, Y.A., Gall, J.: MS-TCN: multi-stage temporal convolutional network for action segmentation. In: IEEE CVPR, pp. 3575–3584 (2019)
Fayyaz, M., Gall, J.: SCT: set constrained temporal transformer for set supervised action segmentation. In: CVPR, pp. 498–507 (2020)
Gattupalli, S., Ghaderi, A., Athitsos, V.: Evaluation of deep learning based pose estimation for sign language recognition. In: PETRA, p. 12. ACM (2016)
Ishikawa, Y., Kasai, S., Aoki, Y., Kataoka, H.: Alleviating over-segmentation errors by detecting action boundaries. In: IEEE WACV, pp. 2321–2330. IEEE (2021)
Johansson, G.: Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14(2), 201–211 (1973)
Li, S., Abu Farha, Y., Liu, Y., Cheng, M.-M., Gall, J.: MS-TCN++: multi-stage temporal convolutional network for action segmentation. CoRR, abs/2006.09220 (2020)
McClelland, M.M., Cameron, C.E.: Self-regulation in early childhood: improving conceptual clarity and developing ecologically valid measures. Child Dev. Perspect. 6(2), 136–142 (2012)
McClelland, M.M., et al.: Predictors of early growth in academic achievement: the head-toes-knees-shoulders task. Front. Psychol. 5(2), 599 (2014)
Song, Y.-F., Zhang, Z., Shan, C., Wang, L.: Stronger, faster and more explainable: a graph convolutional baseline for skeleton-based action recognition. In: MM, pp. 1625–1633. ACM (2020)
Song, Y.-F., Zhang, Z., Shan, C., Wang, L.: Constructing stronger and faster baselines for skeleton-based action recognition. IEEE Trans. Patterns Pattern Anal. Mach. Intell. 45(2), 1474–1488 (2021)
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: CVPR, pp. 5693–5703 (2019)
Tran, D., Wang, H., Torresani, L., Feiszli, M.: Video classification with channel-separated convolutional networks. In: ICCV, pp. 5552–5561 (2019)
Tucha, O.: The history of attention deficit hyperactivity disorder. ADHD Atten. Defic. Hyperact. Disord. 2(4), 241–255 (1999)
Wang, D., Hu, D., Li, X., Dou, D.: Temporal relational modeling with self-supervision for action segmentation. CoRR, abs/2012.07508 (2020)
Wilson, A.D., Golonka, S.: Embodied cognition is not what you think it is. Front. Psychol. 12(2) (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Elmi, S., Bell, M. (2023). Cognitive Assessment Based on Skeleton-Based Action Recognition: Ball-Drop Case. In: Jallouli, R., Bach Tobji, M.A., Belkhir, M., Soares, A.M., Casais, B. (eds) Digital Economy. Emerging Technologies and Business Innovation. ICDEc 2023. Lecture Notes in Business Information Processing, vol 485. Springer, Cham. https://doi.org/10.1007/978-3-031-42788-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-42788-6_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42787-9
Online ISBN: 978-3-031-42788-6
eBook Packages: Computer ScienceComputer Science (R0)