Skip to main content
SpringerLink
Log in

MiTAR: a study on human activity recognition based on NLP with microscopic perspective

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Nowadays, human activity recognition is becoming a more and more significant topic, and there is also a wide range of applications for it in real world scenarios. Sensor data is an important data source in engineering and application. At present, some studies have been carried out in the field of human activity recognition based on sensor data in a macroscopic perspective. However, many studies in this perspective face some limitations. One pivotal limitation is uncontrollable data segment length of different kinds of activities. Suitable feature and data form are also influencing factors. This paper carries out the study creatively on a microscopic perspective with an emphasis on the logic and relevance between data segments, attempting to apply the idea of natural language processing and the method of data symbolization to the study of human activity recognition and try to solve the problem above. In this paper, several activity-element definitions and three algorithms are proposed, including the algorithm of dictionary building, the algorithm of corpus building, and activity recognition algorithm improved from a natural language analysis method, TF-IDF. Numerous experiments on different aspects of this model are taken. The experiments are carried out on six complex and representative single-level sensor datasets, namely UCI Sports and Daily dataset, Skoda dataset, WISDM Phoneacc dataset, WISDM Watchacc dataset, Healthy Older People dataset and HAPT dataset, which prove that this model can be applied to different datasets and achieve a satisfactory recognition result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lin W, Yang M, Wu J, Ke L, Xiong H. Action recognition with coarse-to-fine deep feature integration and asynchronous fusion. In: Proceedings of the National Conference on Artificial Intelligence. 2018, 1–8

  2. Franco R, Facundo Q, Laura L, Cesar E. Distribution of action movements dam a descriptor for human action recognition. Frontiers of Computer Science, 2015, 9(6): 956–965

    Article  Google Scholar 

  3. Chen K, Ding G, Han J. Attribute-based supervised deep learning model for action recognition. Frontiers of Computer Science, 2017, 11(2): 219–229

    Article  Google Scholar 

  4. Wang J, Chen D, Yang J. Human behavior classification by analyzing periodic motions. Frontiers of Computer Science, 2010, 4(4): 580–587

    Article  Google Scholar 

  5. Bracciali A, Larsson E. Data-intensive modelling and simulation in life sciences and socio-economical and physical sciences. Data Science and Engineering, 2017, 2(3): 197–198

    Article  Google Scholar 

  6. Pan W, Li Z, Zhang Y, Weng C. The new hardware development trend and the challenges in data management and analysis. Data Science and Engineering, 2018, 3(3): 263–276

    Article  Google Scholar 

  7. Wu H, Pan W, Xiong X, Xu S. Human activity recognition based on the combined svm&hmm. In: Proceedings of IEEE International Conference on Information & Automation. 2014, 219–224

  8. Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz J L. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: Proceedings of International Conference on Ambient Assisted Living & Home Care. 2012, 216–223

  9. Krishnan R, Subedar M, Tickoo O. Bar: Bayesian activity recognition using variational inference. In: Proceedings of the 3rd Workshop on Bayesian Deep Learning. 2018, 1–8

  10. Dave V S, Zhang B, Chen P, Hasan M A. Neural-brane: neural bayesian personalized ranking for attributed network embedding. Data Science and Engineering, 2019, 4(2): 119–131

    Article  Google Scholar 

  11. Yuan M, Chen E, Lei G. Posture selection based on two-layer AP with application to human action recognition using HMM. In: Proceedings of IEEE International Symposium on Multimedia. 2017, 359–364

  12. Ranjan N, Mundada K, Phaltane K, Ahmad S. A survey on techniques in NLP. International Journal of Computer Applications, 2016, 134: 6–9

    Article  Google Scholar 

  13. Altun K, Barshan B, Tunçel O. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition, 2010, 43(10): 3605–3620

    Article  MATH  Google Scholar 

  14. Zappi P, Stiefmeier T, Farella E, Roggen D, Benini L, Troster G. Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness. In: Proceedings of International Conference on Intelligent Sensors. 2007, 281–286

  15. Roggen D, Calatroni A, Rossi M, Holleczek T, Förster K, Tröster G, Lukowicz P, Bannach D, Pirkl G, Ferscha A. Collecting complex activity data sets in highly rich networked sensor environments. In: Proceedings of the 7th International Conference on Networked Sensing Systems. 2010, 233–240

  16. Chavarriaga R, Sagha H, Calatroni A, Digumarti S, Tröster G, Millán J D R, Roggen D. The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters, 2013, 34(15): 2033–2042

    Article  Google Scholar 

  17. Xie X. Human action recognition in the range of Wi-Fi with CNN and ELM. Master Thesis, Beijing, University of Posts and Telecommunication, 2018

  18. Khanna R, Awad M. Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers. Berkeley California: Apress, 2015

    Google Scholar 

  19. Bharti P, De D, Chellappan S, Das S K. Human: complex activity recognition with multi-modal multi-positional body sensing. IEEE Transactions on Mobile Computing, 2018, 18(4): 857–870

    Article  Google Scholar 

  20. Stolke A, Omohundro S. Hidden markrov model induction by bayesian model merging. In: Proceedings of the 5th International Conference on Neural Information Processing Systems. 1992, 11–18

  21. Adil M K, Young-Koo L, Lee S Y, Tae-Seong K. A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine, 2010, 14(5): 1166–1172

    Article  Google Scholar 

  22. Greff K, Srivastava R K, Koutnik J, Steunebrink B R, Schmidhuber J. LSTM: a search space odyssey. IEEE Transactions on Neural Networks & Learning Systems, 2016, 28(10): 2222–2232

    Article  MathSciNet  Google Scholar 

  23. Qi H, Fang K, Wu X, Xu L, Lang Q. Human activity recognition method based on molecular attributes. International Journal of Distributed Sensor Networks, 2019, 15(4): 1–13

    Article  Google Scholar 

  24. Ashish V, Noam S, Niki P, Jakob U, Llion J, Aidan N G, Lukasz K, Illia P. Attention is all you need. In: Proceedings of Annual Conference on Neural Information Processing Systems. 2017, 5998–6008

  25. Devlin J, Chang M, Lee K, Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 4171–4186

  26. Barshan B, Yüksek M C. Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. Computer Journal, 2013, 57(11): 1649–1667

    Article  Google Scholar 

  27. Brena R, Garcia-Ceja E. A crowdsourcing approach for personalization in human activities recognition. Intelligent Data Analysis, 2017, 21: 721–738

    Article  Google Scholar 

  28. Wang X, Wang L, Lopes L. Unsupervised categorization of human motion sequences. Intelligent Data Analysis, 2013, 17(6): 1057–1074

    Article  Google Scholar 

  29. Wang X, Zhang B, Teng G, Sun Z, Wei J. Toward robust activity recognition: hierarchical classifier based on gaussian process. Intelligent Data Analysis, 2016, 20(3): 701–717

    Article  Google Scholar 

  30. Kantor P. Foundations of statistical natural language processing. Information Retrieval, 2001, 4(1): 80

    Article  Google Scholar 

  31. Khair E L, Ibrahim A. TF*IDF. Boston: Springer US, 2009

    Google Scholar 

  32. Mika S, Schölkopf B, Smola A, Müller K R, Rätsch G. Kernel PCA and De-noising in Feature Spaces. In: Proceedings of the 12th Annual Conference on Neural Information Processing Systems II. 1999, 536–542

  33. Altun K, Barshan B. Human activity recognition using inertial/magnetic sensor units. Lecture Notes in Computer Science, 2010, 6219: 38–51

    Article  Google Scholar 

  34. Zappi P, Lombriser C, Stiefmeier T, Farella E, Roggen D, Benini L, Tröster G. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection. Berlin: Springer Berlin Heidelberg, 2008

    Google Scholar 

  35. Roggen D, Troster G. Fusion of string-matched templates forcontinuous activity recognition. In: Proceedings of IEEE International Symposium on Wearable Computers. 2007, 1–4

  36. Gary M W, Kenichi Y, Thaier H. Smartphone and smartwatch-based biometrics using activities of daily living. IEEE Access, 2019, 7: 133190–133202

    Article  Google Scholar 

  37. Wickramasinghe A, Ranasinghe D C, Fumeaux C, Hill K D, Visvanathan R. Sequence learning with passive rfid sensors for real-time bed-egress recognition in older people. IEEE Journal of Biomedical & Health Informatics, 2017, 21(4): 917–929

    Article  Google Scholar 

  38. Roberto S T, Renuka V, Stephen H, Anton V D H, Damith R. Effectiveness of a batteryless and wireless wearable sensor system for identifying bed and chair exits in healthy older people. Sensors, 2016, 16(4): 546–562

    Article  Google Scholar 

  39. Wickramasinghe A, Ranasinghe D C. Recognising activities in real time using body worn passive sensors with sparse data streams: to interpolate or not to interpolate? In: Proceedings of the 12th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 2015, 21–30

  40. Roberto S T, Damith R, Shi Q. Evaluation of wearable sensor tag data segmentation approaches for real time activity classification in elderly. Springer International Publishing, 2014, 131: 384–395

    Google Scholar 

  41. Sample A P, Roberto S T, Ranasinghe D C, Shi Q. Sensor enabled wearable rfid technology for mitigating the risk of falls near beds. In: Proceedings of IEEE International Conference on RFID. 2013, 191–198

  42. Reyes-Ortiz J L, Oneto L, Sama A, Parra X, Anguita D. Transition-aware human activity recognition using smartphones. Neurocomputing, 2016, 171: 754–767

    Article  Google Scholar 

  43. Trabelsi D, Mohammed S, Amirat Y, Oukhellou L. Activity recognition using body mounted sensors: an unsupervised learning based approach. In: Proceedings of International Joint Conference on Neural Networks. 2012, 1–7

  44. Subasi A. Eeg signal classification using wavelet feature extraction and a mixture of expert model. Expert Systems with Applications, 2007, 32(4): 1084–1093

    Article  Google Scholar 

  45. Li B, Aleksandr D, Gue Y, Liu T, Satoshi M, Du X. Scaling word2vec on big corpus. Data Science and Engineering, 2019, 4(2): 157–175

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by the National Key R&D Program of China (2019YFB1405302), the NSFC (Grant No.61872072), and the State Key Laboratory of Computer Software New Technology Open Project Fund (KFKT2018B05).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Northeastern University, Shenyang, 110000, China

    Huichao Men, Botao Wang & Gang Wu

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210000, China

    Gang Wu

Authors
  1. Huichao Men
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Botao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Botao Wang.

Additional information

Huichao Men is a PhD student of the Department of Computer Science and Engineering at Northeastern University, China now. Her main research interests include time series data analysis, and human activity recognition.

Botao Wang received his PhD in Computer Science in 2000 from Kyushu University, Japan. Currently, he is a professor in the Department of Computer Science and Engineering, Northeastern University, China. His research interests include time series data analysis and privacy protection.

Gang Wu is an associate professor of the School of Computer Science and Engineering at Northeastern University, China. He received his BS and MS degrees from Northeastern University, China in 2000 and 2003 respectively, and his PhD degree from Tsinghua University, China in 2008. His main research interests include main memory database, knowledge graph, and social networks. He is a member of ACM, a member of Chinese Information Processing Society of China, and a member of China Computer Federation.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Men, H., Wang, B. & Wu, G. MiTAR: a study on human activity recognition based on NLP with microscopic perspective. Front. Comput. Sci. 15, 155330 (2021). https://doi.org/10.1007/s11704-020-9495-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-9495-0

Keywords

Search

Navigation