Abstract
Neurobehavioral evidence suggests that human movement may be characterized by relatively stable individual differences (i.e. individual motor signatures or IMS). While most research has focused on the macroscopic level, all attempts to extract IMS have overlooked the fact that functionally relevant discontinuities are clearly visible when zooming into the microstructure of movements. These recurrent (2–3 Hz) speed breaks (sub-movements) reflect an intermittent motor control policy that might provide a far more robust way to identify IMSs.
In this study, we show that individuals can be recognized from motion capture data using a neural network. In particular, we trained a classifier (a convolutional neural network) on a data set composed of time series recording the positions of index finger movements of 60 individuals; in tests, the neural network achieves an accuracy of 80%.
We also investigated how different pre-processing techniques affect the accuracy in order to assess which motion features more strongly characterize each individual and, in particular, whether the presence of submovements in the data can improve the classifier’s performance.
This work was partly supported by the University of Ferrara FIRD 2022 project “Analisi di serie temporali da motion capture con tecniche di machine learning”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018). https://doi.org/10.1109/ACCESS.2018.2870052
Ahmad, M.A., Eckert, C., Teredesai, A.: Interpretable machine learning in healthcare. In: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, BCB 2018, pp. 559–560. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3233547.3233667
Assaf, R., Schumann, A.: Explainable deep neural networks for multivariate time series predictions. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 6488–6490. International Joint Conferences on Artificial Intelligence Organization, Macao (2019). https://doi.org/10.24963/ijcai.2019/932
Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K.: How to explain individual classification decisions, p. 29 (2010)
Burkart, N., Huber, M.F.: A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245–317 (2021). https://doi.org/10.1613/jair.1.12228
Burke, R.E.: Motor units: anatomy, physiology, and functional organization, pp. 345–422. Wiley (2011). https://doi.org/10.1002/cphy.cp010210, https://onlinelibrary.wiley.com/doi/abs/10.1002/cphy.cp010210
Burrell, J.: How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 3(1), 205395171562251 (2016). https://doi.org/10.1177/2053951715622512
Cui, Z., Chen, W., Chen, Y.: Multi-scale convolutional neural networks for time series classification (2016)
Ernst, C.: Artificial intelligence and autonomy: self-determination in the age of automated systems. In: Wischmeyer, T., Rademacher, T. (eds.) Regulating Artificial Intelligence, pp. 53–73. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32361-5_3
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Deep learning for time series classification: a review. Data Min. Knowl. Disc. 33(4), 917–963 (2019). https://doi.org/10.1007/s10618-019-00619-1
Foster, K.R., Koprowski, R., Skufca, J.D.: Machine learning, medical diagnosis, and biomedical engineering research - commentary. Biomed. Eng. Online 13(1), 94 (2014). https://doi.org/10.1186/1475-925X-13-94
Gee, A.H., Garcia-Olano, D., Ghosh, J., Paydarfar, D.: Explaining deep classification of time-series data with learned prototypes, p. 8 (2019)
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 80–89 (2018). https://doi.org/10.1109/DSAA.2018.00018
Gohar, I., et al.: Person re-identification using deep modeling of temporally correlated inertial motion patterns. Sensors 20(3), 949 (2020). https://doi.org/10.3390/s20030949
Goodfellow, S.D., Goodwin, A., Greer, R., Laussen, P.C., Mazwi, M., Eytan, D.: Towards understanding ECG rhythm classification using convolutional neural networks and attention mappings, p. 18 (2018)
Heenaye-Mamode Khan, M., et al.: Multi- class classification of breast cancer abnormalities using deep convolutional neural network (CNN). PLOS One 16(8), 1–15 (2021). https://doi.org/10.1371/journal.pone.0256500
Hu, Y., Sokolova, M.: Convolutional neural networks in multi-class classification of medical data, p. 13 (2020)
Kim, Y.: Convolutional neural networks for sentence classification (2014)
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989). https://doi.org/10.1162/neco.1989.1.4.541
Leventi-Peetz, A.M., Östreich, T.: Deep learning reproducibility and explainable AI (XAI) (2022)
Li, L., Prakash, B.A., Faloutsos, C.: Parsimonious linear fingerprinting for time series. Proc. VLDB Endow. 3(1–2), 385–396 (2010). https://doi.org/10.14778/1920841.1920893
Little, J.J., Boyd, J.E.: Recognizing people by their gait: the shape of motion, p. 33 (1998)
Park, G., Lee, K.M., Koo, S.: Uniqueness of gait kinematics in a cohort study. Sci. Rep. 11(1), 15248 (2021). https://doi.org/10.1038/s41598-021-94815-z
Preece, A.: Asking ‘Why’ in AI: explainability of intelligent systems – perspectives and challenges. Intell. Syst. Account. Financ. Manage. 25(2), 63–72 (2018). https://doi.org/10.1002/isaf.1422
Repp, B.H., Su, Y.-H.: Sensorimotor synchronization: a review of recent research (2006–2012). Psychon. Bull. Rev. 20(3), 403–452 (2013). https://doi.org/10.3758/s13423-012-0371-2
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should i trust you?”: explaining the predictions of any classifier (2016)
Samek, W., Montavon, G., Lapuschkin, S., Anders, C.J., Müller, K.R.: Explaining deep neural networks and beyond: a review of methods and applications. Proc. IEEE 109(3), 247–278 (2021). https://doi.org/10.1109/JPROC.2021.3060483
Selbst, A.D., Powles, J.: Meaningful information and the right to explanation. Int. Data Priv. Law 7(4), 233–242 (2017). https://doi.org/10.1093/idpl/ipx022
Šimić, I., Sabol, V., Veas, E.: XAI methods for neural time series classification: a brief review (2021)
Tomassini, A., et al.: Interpersonal synchronization of movement intermittency. iScience 25(4), 104096 (2022). https://doi.org/10.1016/j.isci.2022.104096
Vale, D., El-Sharif, A., Ali, M.: Explainable artificial intelligence (XAI) post-hoc explainability methods: risks and limitations in non-discrimination law. AI Ethics (2022). https://doi.org/10.1007/s43681-022-00142-y
Woan Ching, S.L., et al.: Multiclass convolution neural network for classification of COVID-19 CT images. Comput. Intell. Neurosci. 2022, 1–15 (2022). https://doi.org/10.1155/2022/9167707
Yang, J.B., Nguyen, M.N., San, P.P., Li, X.L., Krishnaswamy, S.: Deep convolutional neural networks on multichannel time series for human activity recognition, p. 7 (2015)
Zheng, Y., Liu, Q., Chen, E., Ge, Y., Zhao, J.L.: Time series classification using multi-channels deep convolutional neural networks. In: Li, F., Li, G., Hwang, S., Yao, B., Zhang, Z. (eds.) WAIM 2014. LNCS, vol. 8485, pp. 298–310. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08010-9_33
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2921–2929 (2016). https://doi.org/10.1109/CVPR.2016.319
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Galdi, E.M., Alberti, M., D’Ausilio, A., Tomassini, A. (2023). Why Can Neural Networks Recognize Us by Our Finger Movements?. In: Dovier, A., Montanari, A., Orlandini, A. (eds) AIxIA 2022 – Advances in Artificial Intelligence. AIxIA 2022. Lecture Notes in Computer Science(), vol 13796. Springer, Cham. https://doi.org/10.1007/978-3-031-27181-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-27181-6_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27180-9
Online ISBN: 978-3-031-27181-6
eBook Packages: Computer ScienceComputer Science (R0)