Abstract
Continuous human activity recognition from inertial signals is performed by splitting these temporal signals into time windows and identifying the activity in each window. Defining the appropriate window duration has been the target of several previous works. In most of these analyses, the recognition performance increases with the windows duration until an optimal value and decreases or saturates for longer windows. This paper evaluates several strategies to combine sub-window information inside a window, obtaining important improvements for long windows. This evaluation was performed using a state-of-the-art human activity recognition system based on Convolutional Neural Networks (CNNs). This deep neural network includes convolutional layers to learn features from signal spectra and additional fully connected layers to classify the activity at each window. All the analyses were carried out using two public datasets (PAMAP2 and USC-HAD) and a Leave-One-Subject-Out (LOSO) cross-validation. For 10-s windows, the accuracy increased from 90.1 (± 0.66) to 94.27 (± 0.46) in PAMAP2 and from 80.54 (± 0.73) to 84.46 (± 0.67) in USC-HAD. For 20-s windows, the improvements were from 92.66 (± 0.58) to 96.35 (± 0.38) (PAMAP2) and from 78.39 (± 0.76) to 86.36 (± 0.57) (USC-HAD).
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Zhuang Z, Xue Y (2019) Sport-related human activity detection and recognition using a smartwatch. Sensors 19(22):5001. https://doi.org/10.3390/s19225001
Fang H, Hu C (2014) Recognizing Human Activity in Smart Home Using Deep Learning algorithm. In 33rd Chinese Control Conference (CCC), Nanjing, Peoples of Pepublic China, 2014. Jul 28–30 2014, in Chinese Control Conference, pp 4716–4720. [Online]. Available: <Go to ISI>://WOS:000366482804147
Khan AT, Li S, Cao X (2021) Control framework for cooperative robots in smart home using bio-inspired neural network. Measurement 167:108253. https://doi.org/10.1016/j.measurement.2020.108253
Tamoor KA, Shuai L (2021) Human guided cooperative robotic agents in smart home using beetle antennae search. Sci China Inf Sci. https://doi.org/10.1007/s11432-020-3073-5
Gil-Martín M, Montero JM, San-Segundo R (2019) Parkinson’s disease detection from drawing movements using convolutional neural networks. Electronics 8(8):907. https://doi.org/10.3390/electronics8080907 (In English)
Neto et al. (2016) Recognition of Human Activities using the User's Context and the Activity Theory for Risk Prediction. (in English), Proceedings of the 18th International Conference on Enterprise Information Systems, Vol 2 (Iceis), Proceedings Paper pp 282–289 https://doi.org/10.5220/0005832202820289.
Li Z, Zuo W, Li S (2020) Zeroing dynamics method for motion control of industrial upper-limb exoskeleton system with minimal potential energy modulation. Measurement 163:107964. https://doi.org/10.1016/j.measurement.2020.107964
Li Z, Li C, Li S, Cao X (2019) A Fault-tolerant method for motion planning of industrial redundant manipulator. IEEE Trans Ind Informatics 16(12):7469–7478. https://doi.org/10.1109/TII.2019.2957186
Gil-Martín M, Antúnez-Durango J, San-Segundo R (2020) Adaptation and selection techniques based on deep learning for human activity recognition using inertial sensors. Eng Proceed 2(1):22
Gil-Martin M, San-Segundo R, de Cordoba R, Manuel Pardo J (2020) Robust biometrics from motion wearable sensors using a D-vector approach. Neural Process Lett. https://doi.org/10.1007/s11063-020-10339-z
San-Segundo R, Navarro-Hellín H, Torres-Sánchez R, Hodgins J, De la Torre F (2019) Increasing robustness in the detection of freezing of gait in Parkinson’s disease. Electronics 8(2):119. https://doi.org/10.3390/electronics8020119
Lu H, Jin L, Luo X, Liao B, Guo D, Xiao L (2019) RNN for solving perturbed time-varying underdetermined linear system with double bound limits on residual errors and state variables. IEEE Trans Industr Inf 15(11):5931–5942. https://doi.org/10.1109/tii.2019.2909142
Lin CY, Chen BY, Wu WC, Lin EY, Tsai CL (2015) Human Action Recognition Based on Non-negative Matrix Factorization. In Asia-Pacific-Signal-and-Information-Processing-Association Annual Summit and Conference (APSIPA ASC), Hong Kong, Peoples Rebublic of China. In Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, 2015, pp 1091-1093. [Online]. Available: <Go to ISI>://WOS:000382954100206
Luo X, Zhou M, Shang M, Li S, Xia Y (2016) A novel approach to extracting non-negative latent factors from non-negative big sparse matrices. IEEE Access 4:2649–2655. https://doi.org/10.1109/access.2016.2556680
Luo X, Zhou M, Li S, Wu D, Liu Z, Shang M (2021) Algorithms of unconstrained non-negative latent factor analysis for recommender systems. IEEE Trans Big Data 7(1):227–240. https://doi.org/10.1109/tbdata.2019.2916868
Gil-Martín M, San-Segundo R, Fernández-Martínez F, de Córdoba R (2020) Human activity recognition adapted to the type of movement. Comput Electr Eng 88:106822. https://doi.org/10.1016/j.compeleceng.2020.106822
Reiss A, and Stricker D (2012) "Creating and benchmarking a new dataset for physical activity monitoring. Presented at the Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, Heraklion, Crete, Greece, [Online]. Available: https://doi.org/10.1145/2413097.2413148.
Kasnesis P, Patrikakis CZ, Venieris IS (2019) PerceptionNet: a deep convolutional neural network for late sensor fusion. Intell Syst Appl 868:101–119. https://doi.org/10.1007/978-3-030-01054-6_7
Ignatov A (2018) Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl Soft Comput 62:915–922. https://doi.org/10.1016/j.asoc.2017.09.027
San-Segundo R, Echeverry-Correa JD, Salamea C, Pardo JM (2016) Human activity monitoring based on hidden markov models using a smartphone. IEEE Instrum Measure Magazine 19(6):27–31. https://doi.org/10.1109/mim.2016.7777649
Khaire P, Kumar P, Imran J (2018) Combining CNN streams of RGB-D and skeletal data for human activity recognition. Pattern Recogn Lett 115:107–116. https://doi.org/10.1016/j.patrec.2018.04.035
Abdel-Salam R, Mostafa R, and Hadhood M (2021) Human activity recognition using wearable sensors: review, challenges, evaluation benchmark. p. arXiv:2101.01665
Tanjid Hasan Tonmoy M, Mahmud S, Mahbubur Rahman AKM, Ashraful Amin M, and Ali AA, Hierarchical self attention based autoencoder for open-set human activity recognition. p. arXiv:2103.04279
Mahmud S et al. (2020) Human activity recognition from wearable sensor data using self-attention. p. arXiv:2003.09018
Khan AT, Cao X, Li S, Hu B, Katsikis VN (2021) Quantum beetle antennae search: a novel technique for the constrained portfolio optimization problem. Sci China-Inf Sci 64(5):152204. https://doi.org/10.1007/s11432-020-2894-9
Banos O, Galvez J-M, Damas M, Pomares H, Rojas I (2014) Window size impact in human activity recognition. Sensors 14(4):6474–6499. https://doi.org/10.3390/s140406474
Pirttikangas S, Fujinami K, Nakajima T (2006) Feature selection and activity recognition from wearable sensors. Ubiquitous Comput Syst, Proceed 4239:516–527
Stikic M, Huynh T, Van Laerhoven K, Schiele B (2008) ADL Recognition Based on the Combination of RFID and Accelerometer Sensing. 2008 2nd International Conference on Pervasive Computing Technologies for Healthcare. pp 245
Sun L, Zhang D, Li B, Guo B, and Li S (2010) Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations. Ubiquitous Intelligence and Computing, vol. 6406, p 548
Mannini A, Intille SS, Rosenberger M, Sabatini AM, Haskell W (2013) Activity recognition using a single accelerometer placed at the wrist or ankle. Med Sci Sports Exerc 45(11):2193–2203. https://doi.org/10.1249/MSS.0b013e31829736d6
Mehrang S, Pietila J, Korhonen I (2018) An activity recognition framework deploying the random forest classifier and a single optical heart rate monitoring and Triaxial accelerometer wrist-band. Sensors 18(2):613. https://doi.org/10.3390/s18020613
Zhang M, Sawchuk AA, and Assoc Comp M (2012) USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. Ubicomp'12: Proceedings of the 2012 Acm International Conference on Ubiquitous Computing pp 1036–1043
Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv (CSUR) 46(3):1–33. https://doi.org/10.1145/2499621
Weiss NA (2017) Introductory statistics. Pearson, London
Roggen D, Calatroni A, Rossi M, Holleczek T, and Forster K (2010) Collecting complex activity datasets in highly rich networked sensor environments. 2010 Seventh International Conference on Networked Sensing Systems (INSS 2010), pp 233–40 https://doi.org/10.1109/inss.2010.5573462.
Funding
The work leading to these results has been supported by AMIC (MINECO, TIN2017-85854-C4-4-R), and CAVIAR (MINECO, TEC2017-84593-C2-1-R) projects partially funded by the European Union. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research.
Author information
Authors and Affiliations
Contributions
M. Gil-Martín: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Roles/Writing—original draft. R. San-Segundo: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing—review & editing. F. Fernández-Martínez: Conceptualization, Funding acquisition, Project administration, Supervision, Validation, Writing—review & editing. J. Ferreiros-López: Conceptualization, Funding acquisition, Project administration, Supervision, Validation, Writing—review & editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gil-Martín, M., San-Segundo, R., Fernández-Martínez, F. et al. Time Analysis in Human Activity Recognition. Neural Process Lett 53, 4507–4525 (2021). https://doi.org/10.1007/s11063-021-10611-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-021-10611-w