Skip to main content
SpringerLink
Log in

Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In recent years, with the rapid development of artificial intelligence, human activity recognition has become a research focus. The complex, dynamic and variable features of human activities lead to the relatively low accuracy of the traditional recognition algorithms. In order to solve the problem, this paper will propose a novel structure named hierarchical deep LSTM (H-LSTM) based on long short-term memory. Firstly, the original sensor data are preprocessed by smoothing and denoising; then, the feature will be selected and extracted by time–frequency-domain method. Secondly, H-LSTM is applied to the classification of these activities. Three public UCI datasets are used to conduct simulation with the realization of the automatic extraction of feature vectors and classification of outputting recognition results. Finally, the simulation results testify to the outperformance of the H-LSTM network over other deep learning algorithms. The accuracy of H-LSTM network in human activity recognition is proved to be 99.15%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. S. Ali, N.A. Khan, M. Haneef, X.J.C. Luo, Blind source separation schemes for mono-sensor and multi-sensor systems with application to signal detection. Circuits Syst. Signal Process. 36(11), 4615–4636 (2017)

    Article  MathSciNet  Google Scholar 

  2. K. Altun, B. Barshan, O. Tunçel, Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recogn. 43(10), 3605–3620 (2010)

    Article  Google Scholar 

  3. D. Anguita, A. Ghio, L. Oneto, X. Parra, J.L. Reyes-Ortiz, Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine, in International Workshop on Ambient Assisted Living (Springer, Berlin, 2012), pp. 216–223

    Google Scholar 

  4. J. Azorin-Lopez, M. Saval-Calvo, A. Fuster-Guillo, J. Garcia-Rodriguez, A novel prediction method for early recognition of global human behaviour in image sequences. Neural Process. Lett. 43(2), 363–387 (2015)

    Article  Google Scholar 

  5. M. Babiker, O.O. Khalifa, K.K. Htike, A. Hassan, M. Zaharadeen, Automated daily human activity recognition for video surveillance using neural network, in Proceedings of 2017 4th IEEE International Conference on Smart Instrumentation, Measurement and Application (IEEE, 2017), pp. 1–5

  6. O. Banos, J.-M. Galvez, M. Damas, A. Guillen, L.-J. Herrera, H. Pomares, I. Rojas, C. Villalonga, C.S. Hong, S. Lee, Multiwindow fusion for wearable activity recognition, in International Work-Conference on Artificial Neural Networks (Springer, Berlin, 2015), pp. 290–297

    Chapter  Google Scholar 

  7. A. Bulling, U. Blanke, B. Schiele, A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3), 33 (2014)

    Article  Google Scholar 

  8. P. Casale, O. Pujol, P. Radeva, Human activity recognition from accelerometer data using a wearable device, in Proceedings of Iberian Conference on Pattern Recognition and Image Analysis (Springer, Berlin, 2011), pp. 289–296

    Chapter  Google Scholar 

  9. L. Chen, J. Hoey, C.D. Nugent, D.J. Cook, Z. Yu, Sensor-based activity recognition. IEEE Trans. Syst. Man, Cybern. C, Appl. Rev. 42(6), 790–808 (2012)

    Article  Google Scholar 

  10. L. Chen, H. Wei, J. Ferryman, ReadingAct RGB-D action dataset and human action recognition from local features. Pattern Recogn. Lett. 50, 159–169 (2014)

    Article  Google Scholar 

  11. W.-H. Chen, C.A.B. Baca, C.-H. Tou, LSTM-RNNs combined with scene information for human activity recognition, in 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (2017), pp. 1–6

  12. M. Ciman, M. Donini, O. Gaggi, F. Aiolli, Stairstep recognition and counting in a serious game for increasing users’ physical activity. Pers. Ubiquitous Comput. 20(6), 1015–1033 (2016)

    Article  Google Scholar 

  13. K. Cui, X. Jing, Research on prediction model of geotechnical parameters based on BP neural network. Neural. Comput. Appl. (2018). https://doi.org/10.1007/s00521-018-3902-6

    Article  Google Scholar 

  14. M. Edel, E. Köppe, Binarized-blstm-rnn based human activity recognition, in Proceedings of 2016 International Conference on Indoor Positioning and Indoor Navigation (IEEE, 2016), pp. 1–7

  15. P. Esfahani, H.T. Malazi, PAMS: a new position-aware multi-sensor dataset for human activity recognition using smartphones, in Proceedings of 2017 19th IEEE International Symposium on Computer Architecture and Digital Systems (IEEE, 2017), pp. 1–7

  16. L. Fan, Z. Wang, H. Wang, Human activity recognition model based on decision tree, in Proceedings of 2013 International Conference on Advanced Cloud and Big Data (IEEE, 2013), pp. 64–68

  17. F. Gu, K. Khoshelham, S. Valaee, J. Shang, R. Zhang, Locomotion activity recognition using stacked denoising autoencoders. IEEE Internet Things. 5(3), 2085–2093 (2018)

    Article  Google Scholar 

  18. P. Gupta, T. Dallas, Feature selection and activity recognition system using a single triaxial accelerometer. IEEE Trans. Biomed. Eng. 61(6), 1780–1786 (2014)

    Article  Google Scholar 

  19. H.P. Gupta, H.S. Chudgar, S. Mukherjee, T. Dutta, K. Sharma, A continuous hand gestures recognition technique for human–machine interaction using accelerometer and gyroscope sensors. IEEE Sens. J. 16(16), 6425–6432 (2016)

    Article  Google Scholar 

  20. S. Hochreiter, J. Schmidhuber, LSTM can solve hard long time lag problems, in Advances in Neural Information Processing Systems (1997), pp. 473–479

  21. A. Jain, V. Kanhangad, Investigating gender recognition in smart-phones using accelerometer and gyroscope sensor readings, in Proceedings of 2016 International Conference on Computational Techniques in Information and Communication Technologies (2016), pp. 597–602

  22. D. Li, H. Zhang, M.J.C. Zhang, Wavelet de-noising and genetic algorithm-based least squares twin SVM for classification of arrhythmias. Circuits Syst. Signal Process. 36(7), 2828–2846 (2017)

    Article  MathSciNet  Google Scholar 

  23. C.L. Liu, C.H. Lee, P.M. Lin, A fall detection system using k-nearest neighbor classifier. Expert Syst. Appl. 37(10), 7174–7181 (2010)

    Article  Google Scholar 

  24. B. Long, M. Li, H. Wang, S.J.C. Tian, Diagnostics of analog circuits based on LS-SVM using time-domain features. Circuits Syst. Signal Process. 32(6), 2683–2706 (2013)

    Article  MathSciNet  Google Scholar 

  25. Y. Lu, Y. Wei, L. Liu, J. Zhong, L. Sun, Y. Liu, Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed. Tools Appl. 76(8), 10701–10719 (2017)

    Article  Google Scholar 

  26. D. Micucci, M. Mobilio, P. Napoletano, Unimib shar: a dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7(10), 1101 (2017)

    Article  Google Scholar 

  27. M. Milenkoski, K. Trivodaliev, S. Kalajdziski, M. Jovanov, B.R. Stojkoska, Real time human activity recognition on smartphones using LSTM Networks, in Proceedings of 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (IEEE, 2018), pp. 1126–1131

  28. U.M. Nunes, D.R. Faria, P. Peixoto, A human activity recognition framework using max-min features and key poses with differential evolution random forests classifier. Pattern Recogn. Lett. 99, 21–31 (2017)

    Article  Google Scholar 

  29. F. Ordóñez, D. Roggen, Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors. 16(1), 115 (2016)

    Article  Google Scholar 

  30. J. Park, K. Jang, S.-B. Yang, Deep neural networks for activity recognition with multi-sensor data in a smart home, in 2018 4th IEEE World Forum on Internet of Things (IEEE, 2018), pp. 155–160

  31. S.U. Park, J.H. Park, M.A. Al-masni, M.A. Al-antari, M.Z. Uddin, T.S. Kim, A depth camera-based human activity recognition via deep learning recurrent neural network for health and social care services. Proced. Comput. Sci. 100, 78–84 (2016)

    Article  Google Scholar 

  32. S.Y. Park, H. Ju, C.G. Park, Stance phase detection of multiple actions for military drill using foot-mounted IMU. Sensors. 14, 16 (2016)

    Google Scholar 

  33. J.S. Peng, Y.M. Shao, Intelligent method for identifying driving risk based on V2V multisource big data. Complexity. 2018(1), 1–9 (2018)

    MATH  Google Scholar 

  34. S.J. Preece, J.Y. Goulermas, L.P. Kenney, D. Howard, A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Trans. Biomed. Eng. 56(3), 871–879 (2009)

    Article  Google Scholar 

  35. J.L. Reyes-Ortiz, L. Oneto, A. Ghio, A. Samá, D. Anguita, X. Parra, Human activity recognition on smartphones with awareness of basic activities and postural transitions, in Proceedings of International Conference on Artificial Neural Networks (Springer, Berlin, 2014), pp. 177–184

    Chapter  Google Scholar 

  36. G. Sridevi, S.J.C. Srinivas Kumar, Image inpainting based on fractional-order nonlinear diffusion for image reconstruction. Circuits Syst. Signal Process (2019). https://doi.org/10.1007/s00034-019-01029-w

    Article  Google Scholar 

  37. A. Subasi, M. Radhwan, R. Kurdi, K. Khateeb, IoT based mobile healthcare system for human activity recognition, in Proceedings of 2018 15th IEEE Conference on Learning and Technology (IEEE, 2018), pp. 29–34

  38. P. Susarla, U. Agrawal, D.B. Jayagopi, Human weapon-activity recognition in surveillance videos using structural-RNN, in Proceedings of 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (ACM, New York, 2018), pp. 101–107

  39. G. Vavoulas, M. Pediaditis, E.G. Spanakis, M. Tsiknakis, The MobiFall dataset: An initial evaluation of fall detection algorithms using smartphones, in Proceedings of 13th IEEE International Conference on BioInformatics and BioEngineering (2013), pp. 1–4

  40. T.H. Vu, A. Dang, L. Dung, J.-C. Wang, Self-gated recurrent neural networks for human activity recognition on wearable devices, in Proceedings of the on Thematic Workshops of ACM Multimedia 2017 (ACM, New York, 2017), pp. 179–185

  41. L. Wang, Recognition of human activities using continuous autoencoders with wearable sensors. Sensors. 16(2), 189 (2016)

    Article  Google Scholar 

  42. L. Wang, Three-dimensional convolutional restricted Boltzmann machine for human activity recognition from RGB-D video. Eurasip J. Image Video Process. 2018(1), 120 (2018)

    Article  Google Scholar 

  43. J.H. Wu, W. Wei, L. Zhang et al., Risk assessment of hypertension in steel workers based on LVQ and fisher-SVM deep excavation. IEEE Access. 7, 23109–23119 (2019)

    Article  Google Scholar 

  44. F. Xiao, J. Chen, X.H. Xie, L. Gui, J.L. Sun, W. none Ruchuan, SEARE: a system for exercise activity recognition and quality evaluation based on green sensing. IEEE Trans. Emerg. Top. Comput. (2018). https://doi.org/10.1109/TETC.2018.2790080

    Article  Google Scholar 

  45. A. Yassine, S. Singh, A.J.I.A. Alamri, Mining human activity patterns from smart home big data for health care applications. IEEE Access. 5, 13131–13141 (2017)

    Article  Google Scholar 

  46. H. Zhang, A.C. Berg, M. Maire, J. Malik, SVM-KNN: Discriminative nearest neighbor classification for visual category recognition, in Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) (IEEE, 2006), pp. 2126–2136

  47. X. Zhang, Y. Li, R. Kotagiri, L. Wu, Z. Tari, M. Cheriet, KRNN: k rare-class nearest neighbour classification. Pattern Recogn. 62, 33–44 (2017)

    Article  Google Scholar 

  48. X.Y. Zhang, F. Yin, Y.M. Zhang, C.L. Liu, Y. Bengio, Drawing and recognizing Chinese characters with recurrent neural network. IEEE Trans. Pattern. Anal. 40(4), 849–862 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of Shandong Province (ZR2018BF005), the Scientific Research Foundation of Shandong University of Science and Technology for Recruited Talents (2017RCJJ077), the Shandong Province Higher Educational Science and Technology Program (J17KB167) and the Science and Technology Program of Taian (2017GX0014, 2018ZC0284).

Author information

Authors and Affiliations

  1. Department of Information Engineering, Shandong University of Science and Technology, Taian, 271019, China

    LuKun Wang & RuYue Liu

  2. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, 266590, China

    LuKun Wang

Authors
  1. LuKun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. RuYue Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to LuKun Wang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Liu, R. Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks. Circuits Syst Signal Process 39, 837–856 (2020). https://doi.org/10.1007/s00034-019-01116-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-019-01116-y

Keywords

Search

Navigation