Abstract
With the advances in smart home technology and Internet of Things (IoT), there has been keen research interest in human activity recognition to allow service systems to understand human intentions. Recognizing human objectives by these systems without user intervention, results in better service, which is crucial to improve the user experience. Existing research approaches have focused primarily on probabilistic methods like Bayesian networks (for instance, the CRAFFT algorithm). Though quite versatile, these probabilistic models may be unable to successfully capture the possibly complex relationships between the input variables. To the best of our knowledge, a statistical study of features in a human activity recognition task, their relationships, etc., has not yet been attempted. To this end, we study the domain of human activity recognition to improve the state-of-the-art and present a novel neural network architecture for the task. It employs early fusion on different types of minimalistic features such as time and location to make extremely accurate predictions with a maximum micro F1-score of 0.98 on the Aruba CASAS dataset. We also accompany the model with a comprehensive study of the features. Using feature selection techniques like Leave-One-Out, we rank the features according to the information they add to deep learning models and make further inferences using the ranking obtained. Our empirical results show that the feature Previous Activity Performed is the most useful of all, surprisingly even more than time (the basis of activity scheduling in most societies). We use three Activities of Daily Living (ADL) datasets in different settings to empirically demonstrate the utility of our architecture. We share our findings along with the models and the source code.
A. Singhal and M. Goyal—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Alam, M.R., Reaz, M.B.I., Mohd Ali, M.A.: Speed: an inhabitant activity prediction algorithm for smart homes. IEEE Trans. Syst. Man Cybern. - Part A: Syst. Hum. 42(4), 985–990 (2012). https://doi.org/10.1109/TSMCA.2011.2173568
Ali, F.M., Lee, S.W., Bien, Z., Mokhtari, M.: Combined fuzzy state q-learning algorithm to predict context aware user activity under uncertainty in assistive environment. In: 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, pp. 57–62 (2008). https://doi.org/10.1109/SNPD.2008.13
Chen, C., Das, B., Cook, D.: Energy prediction based on resident’s activity (2010)
Choi, S., Kim, E., Oh, S.: Human behavior prediction for smart homes using deep learning. In: 2013 IEEE RO-MAN, pp. 173–179 (2013)
Cook, D.: Learning setting-generalized activity models for smart spaces. IEEE Intell. Syst. 27, 32–38 (2012)
Cook, D., Schmitter-Edgecombe, M.: Assessing the quality of activities in a smart environment. Methods Inf. Med. 48(5), 480–5 (2009)
Das, S.K., Cook, D.J., Battacharya, A., Heierman, E.O., Lin, T.-Y.: The role of prediction algorithms in the MavHome smart home architecture. IEEE Wirel. Commun. 9(6), 77–84 (2002). https://doi.org/10.1109/MWC.2002.1160085
Goel, P., Goyal, M., Shah, R.: Arten-net: an emotion classification system for art (student consortium). In: 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM), pp. 302–306 (2020)
Jakkula, V., Cook, D.: Mining sensor data in smart environment for temporal activity prediction (2007)
Jouini, R., Lemlouma, T., Maalaoui, K., Saidane, L.A.: Employing grey model forecasting GM(1,1) to historical medical sensor data towards system preventive in smart home e-health for elderly person. In: 2016 International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 1086–1091 (2016). https://doi.org/10.1109/IWCMC.2016.7577210
Liu, K., Li, Y., Xu, N., Natarajan, P.: Learn to combine modalities in multimodal deep learning. ArXiv abs/1805.11730 (2018)
Minor, B., Cook, D.J.: Regression tree classification for activity prediction in smart homes. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp 2014 Adjunct, p. 441–450. Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2638728.2641669
Nazerfard, E., Cook, D.J.: CRAFFT: an activity prediction model based on Bayesian networks. J. Ambient Intell. Humaniz. Comput. 6(2), 193–205 (2014). https://doi.org/10.1007/s12652-014-0219-x
Petscharnig, S., Schöffmann, K., Benois-Pineau, J., Chaabouni, S., Keckstein, J.: Early and late fusion of temporal information for classification of surgical actions in laparoscopic gynecology. In: 2018 IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), pp. 369–374 (2018). https://doi.org/10.1109/CBMS.2018.00071
Rao, S.P., Cook, D.J.: Identifying tasks and predicting actions in smart homes using unlabeled data. In: Proceedings of the Machine Learning Workshop on the Continuum from Labeled to Unlabeled Data (2003)
Seeland, M., Rzanny, M., Alaqraa, N., Wäldchen, J., Mäder, P.: Plant species classification using flower images–a comparative study of local feature representations. PLOS ONE 12, e0170629 (2017). https://doi.org/10.1371/journal.pone.0170629
Shuai, Z., Oh, S., Yang, M.H.: Traffic modeling and prediction using camera sensor networks. In: Proceedings of the Fourth ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC 2010, pp. 49–56. Association for Computing Machinery, New York (2010). https://doi.org/10.1145/1865987.1865996
Vintan, L., Gellert, A., Petzold, J., Ungerer, T.: Person movement prediction using neural networks (2004)
Kang, W., Shin, D., Shin, D.: Detecting and predicting of abnormal behavior using hierarchical Markov model in smart home network. In: 2010 IEEE 17Th International Conference on Industrial Engineering and Engineering Management, pp. 410–414 (2010). https://doi.org/10.1109/ICIEEM.2010.5646583
Wu, E., Zhang, P., Lu, T., Gu, H., Gu, N.: Behavior prediction using an improved hidden Markov model to support people with disabilities in smart homes. In: 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 560–565 (2016). https://doi.org/10.1109/CSCWD.2016.7566051
Wu, Z., Liu, A., Zhou, P., Su, Y.F.: A Bayesian network based method for activity prediction in a smart home system. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 001496–001501 (2016). https://doi.org/10.1109/SMC.2016.7844449
Xu, G., Liu, M., Li, F., Zhang, F., Shen, W.: User behavior prediction model for smart home using parallelized neural network algorithm. In: 2016 IEEE 20th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 221–226 (2016). https://doi.org/10.1109/CSCWD.2016.7565992
Acknowledgement
Raghava Mutharaju and Jainendra Shukla are partly supported by the Infosys Center for AI (CAI) and Jainendra Shukla is also partly supported by the Center for Design and New Media at IIIT-Delhi.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Singhal, A., Goyal, M., Shukla, J., Mutharaju, R. (2021). Feature Fused Human Activity Recognition Network (FFHAR-Net). In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2021 - Posters. HCII 2021. Communications in Computer and Information Science, vol 1420. Springer, Cham. https://doi.org/10.1007/978-3-030-78642-7_72
Download citation
DOI: https://doi.org/10.1007/978-3-030-78642-7_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78641-0
Online ISBN: 978-3-030-78642-7
eBook Packages: Computer ScienceComputer Science (R0)