Skip to main content

Advertisement

SpringerLink
Log in

Sequential neural networks for multi-resident activity recognition in ambient sensing smart homes

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Advances in smart home technology and IoT devices had made us capable of monitoring human activities in a non-intrusive way. This data, in turn, enables us to predict the health status and energy consumption patterns of residents of these smart homes. Machine learning has been an excellent tool for the prediction of human activities from raw sensor data of a single resident. However, Multi Resident activity recognition is still a challenging task, as there is no correlation between sensor values and resident activities. In this paper, we have applied deep learning algorithms on the real world ARAS Multi Resident data set, which consists of data from two houses, each with two residents. We have used different variations of Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and their combination on the data set and kept the labels separate for both residents. We have evaluated the performance of models based on several metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Cook D J, Das S, Gopalratnam K, Roy A (2003) Health monitoring in an agent-based smart home. In: Proceedings of the International Conference on Aging, Disability and Independence Advancing Technology and Services to Promote Quality of Life, pp 3–141

  2. Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28 (6):976–990

    Article  Google Scholar 

  3. Plötz T, Hammerla N Y, Olivier P L (2011) Feature learning for activity recognition in ubiquitous computing. In: Twenty-Second International Joint Conference on Artificial Intelligence

  4. Shelke S, Aksanli B (2019) Static and dynamic activity detection with ambient sensors in smart spaces. Sensors 19(4):804

    Article  Google Scholar 

  5. Using a hidden markov model for resident identication (2010) International Conference on Intelligent Environments, pp 7479–7479

  6. Vail D L, Veloso M M, Lafferty J D (2007) Conditional random fields for activity recognition. In: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems. ACM, pp 235

  7. Prossegger M, Bouchachia H (2014) Multi-resident activity recognition using incremental decision trees, pp 182–191

  8. Xu L, Yang W, Cao Y, Li Q (2017) Human activity recognition based on random forests. In: 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). IEEE, pp 548–553

  9. Putra DNS, Yulita IN (2019) Multilayer perceptron for activity recognition using a batteryless wearable sensor. In: IOP Conference Series: Earth and Environmental Science, vol 248. IOP Publishing, pp 012039

  10. Singh D, Merdivan E, Psychoula I, Kropf J, Hanke S, Geist M, Holzinger A (2017) Human activity recognition using recurrent neural networks. In: International Cross-Domain Conference for Machine Learning and Knowledge Extraction. Springer, pp 267–274

  11. Ji S, Xu W, Yang M, Yu K (2012) 3d convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231

    Article  Google Scholar 

  12. Alemdar H, Ertan H, Incel O D, Ersoy C (2013) Aras human activity datasets in multiple homes with multiple residents, pp 232–235

  13. Jordao A, Nazare Jr A C, Sena J, Schwartz W R (2018) Human activity recognition based on wearable sensor data: A standardization of the state-of-the-art. arXiv:1806.05226

  14. Zubair M, Song K, Yoon C (2016) Human activity recognition using wearable accelerometer sensors. IEEE, pp 1–5

  15. Zhang S, Wei Z, Nie J, Huang L, Wang S, Li Z (2017) A review on human activity recognition using vision-based method. Journal of Healthcare Engineering 2017

  16. Cook D J, Crandall A S, Thomas B L, Krishnan N C (2012) Casas: A smart home in a box. Computer 46(7):62–69

    Article  Google Scholar 

  17. Ye J, Stevenson G, Dobson S (2015) Kcar: A knowledge-driven approach for concurrent activity recognition. Pervasive Mob Comput 19:47–70

    Article  Google Scholar 

  18. Shet V D, Harwood D, Davis L S (2005) Vidmap: video monitoring of activity with prolog. In: IEEE Conference on Advanced Video and Signal Based Surveillance. IEEE, pp 224–229

  19. Artikis A, Sergot M, Paliouras G (2013) A logic-based approach to activity recognition. In: Human Behavior Recognition Technologies: Intelligent Applications for Monitoring and Security. IGI Global, pp 1–13

  20. Kowalski R, Sergot M (1989) A logic-based calculus of events. In: Foundations of knowledge base management. Springer, pp 23–55

  21. Cook D J (2010) Learning setting-generalized activity models for smart spaces. IEEE Intell Syst 2010(99):1

    Google Scholar 

  22. Cook D J, Krishnan N C, Rashidi P (2013) Activity discovery and activity recognition: A new partnership. IEEE Trans Cybern 43(3):820–828

    Article  Google Scholar 

  23. Fahad L G, Tahir S F, Rajarajan M (2014) Activity recognition in smart homes using clustering based classification. In: 2014 22nd International Conference on Pattern Recognition. IEEE, pp 1348–1353

  24. Tran S, Zhang Q, Karunanithi M (2009) Mixed-dependency models for multi-resident activity recognition in smart-homes

  25. Chen R, Tong Y (2014) A two-stage method for solving multi-resident activity recognition in smart environments. Entropy 16(4):2184–2203

    Article  Google Scholar 

  26. Nazerfard E, Das B, Holder L B, Cook D J (2010) Conditional random fields for activity recognition in smart environments. In: Proceedings of the 1st ACM International Health Informatics Symposium. ACM, pp 282–286

  27. Hsu K-C, Chiang Y-T, Lin G-Y, Lu C-H, Hsu J Y-J, Fu L-C (2010) Strategies for inference mechanism of conditional random fields for multiple-resident activity recognition in a smart home. In: International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Springer, pp 417–426

  28. Zhuang X, Huang J, Potamianos G, Hasegawa-Johnson M (2009) Acoustic fall detection using gaussian mixture models and gmm supervectors. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp 69–72

  29. Ribaric S, Hrkac T (2012) A model of fuzzy spatio-temporal knowledge representation and reasoning based on high-level petri nets. Inf Syst 37(3):238–256

    Article  Google Scholar 

  30. Tran S N, Nguyen D, Ngo T-S, Vu X-S, Hoang L, Zhang Q, Karunanithi M (2019) On multi-resident activity recognition in ambient smart-homes. Artif Intell Rev:1–17

  31. Tran S N, Zhang Q (2020) Towards multi-resident activity monitoring with smarter safer home platform. In: Smart Assisted Living: Toward An Open Smart-Home Infrastructure. Springer International Publishing, Cham, pp 249–267

  32. Al Machot F, Mosa A, Ali M, Kyamakya K (2017) Activity recognition in sensor data streams for active and assisted living environments. IEEE Trans Circ Syst for Video Technol PP. https://doi.org/10.1109/TCSVT.2017.2764868

  33. Hassan M M, Uddin M Z, Mohamed A, Almogren A (2018) A robust human activity recognition system using smartphone sensors and deep learning. Futur Gener Comput Syst 81:307–313

    Article  Google Scholar 

  34. Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: A survey. Pattern Recogn Lett 119:3–11

    Article  Google Scholar 

  35. Phyo C N, Zin T T, Tin P (2019) Deep learning for recognizing human activities using motions of skeletal joints. IEEE Trans Consum Electron 65(2):243–252. https://doi.org/10.1109/TCE.2019.2908986

    Article  Google Scholar 

  36. Baccouche M, Mamalet F, Wolf C, Garcia C, Baskurt A (2011) Sequential deep learning for human action recognition. In: Salah A A, Lepri B (eds) Human Behavior Understanding. Springer, Berlin, pp 29–39

  37. Li X, Zhang Y, Marsic I, Sarcevic A, Burd R S (2016) Deep learning for rfid-based activity recognition. In: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM. ACM, pp 164–175

  38. Wang K, Wang X, Lin L, Wang M, Zuo W (2014) 3d human activity recognition with reconfigurable convolutional neural networks. In: Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp 97–106

  39. Liciotti D, Bernardini M, Romeo L, Frontoni E (2019) A sequential deep learning application for recognising human activities in smart homes. Neurocomputing

  40. Mohamed R (2017) Multi label classification on multi resident in smart home using classifier chains. Adv Sci Lett 4:400–407

    Google Scholar 

  41. Mohamed R, Perumal T, Sulaiman M, Mustapha N (2017) Multi-resident activity recognition using label combination approach in smart home environment. In: 2017 IEEE International Symposium on Consumer Electronics (ISCE). IEEE, pp 69–71

  42. LeCun Y, Bottou L, Bengio Y, Haffner P, et al. (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  43. Sherstinsky A (2018) Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. arXiv:1808.03314

  44. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  45. Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2625–2634

Download references

Author information

Authors and Affiliations

  1. Electronics and Communication, The LNM Institute of Information Technology, Jaipur, Rajasthan, 302031, India

    Anubhav Natani & Abhishek Sharma

  2. Computer Science Engineering, Universiti Putra Malaysia, Serdang, Selangor, Malaysia

    Thinagaran Perumal

Authors
  1. Anubhav Natani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abhishek Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thinagaran Perumal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Abhishek Sharma or Thinagaran Perumal.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Natani, A., Sharma, A. & Perumal, T. Sequential neural networks for multi-resident activity recognition in ambient sensing smart homes. Appl Intell 51, 6014–6028 (2021). https://doi.org/10.1007/s10489-020-02134-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02134-z

Keywords

Search

Navigation