Skip to main content
SpringerLink
Log in

Suspicious activity detection using deep learning in secure assisted living IoT environments

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Children who are left alone in environments such as daycares and crèches require constant monitoring and care to protect them from abuse. In this paper, we propose a novel deep learning-based method for predicting the occurrence of abnormal events using footage gathered from networked surveillance systems and notifying users of those events in an Internet of Things (IoT) environment. Sequences of images are converted to still frames and de-blurred using adaptive motion detection techniques. Then, abnormal activities are predicted using random forest differential evolution with kernel density (RFKD), and any abnormal activities that are detected cause signals to be sent to IoT devices via the MQTT protocol. The proposed work consists of a multi-classifier, deep neural network and kernel density functions. The multi-classifier is used for input classifications from the sequence of frames of videos. The deep neural network is used to learn and train the data and kernel density is used clustering and prediction of data. The novelty of the proposed work is in the dynamic nature of activity prediction. Most of the previous work in this research area concentrated on static activity prediction. The proposed work is able to support both static and dynamic activities of daycare environments. In our experimental trials, our novel method’s performance is shown to be superior to that of the ReHAR method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. https://www.childlineindia.org.in/pdf/MWCD-Child-Abuse-Report.pdf

  2. http://victimsofcrime.org/media/reporting-on-child-sexual-abuse/child-sexual-abuse-statistics

  3. Marco Stolpe, Dortmund TU (2016) The internet of things: opportunities and challenges for distributed data analysis. Newsl ACM SIGKDD Explor Newsl Arch 18(1):15–34

    Article  Google Scholar 

  4. Verizon. State of the Market: The Internet of Things 2015

  5. Chen L, Hoey J, Nugent C, Cook D, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C 42(6):790–808

    Article  Google Scholar 

  6. Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W. (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, CoBuild’99. Springer, London, pp 191–198

  7. Kasteren TLM, Englebienne G, Krose BJA (2011) Human activity recognition from wireless sensor network data: bench-mark and software. In: Activity Recognition in Pervasive Intelligent Environments. Atlantis Press, Paris, pp 165–186

  8. Cook DJ, Augusto JC, Jakkula VR (2009) Ambient intelligence: technologies, applications, and opportunities. Pervasive Mob Comput 54:277–298

    Article  Google Scholar 

  9. Lloret J, Canovas A, Sendra S, Parra L (2015) Smart communication architecture for ambient assisted living. IEEE Commun Mag 53:26–33

    Article  Google Scholar 

  10. Singh A, Bianchi-Berthouze N, Williams ACC (2017) Supporting everyday function in chronic pain using wearable technology. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11, pp 3903–3915

  11. Ghayvat H, Mukhopadhyay S, Shenjie B, Chouhan A, Chen W (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA, pp 14–17

  12. Martinez-Hernandez U, Dehghani-Sanij AA (2018) Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor. Pattern Recognit Lett 118:32–41

    Article  Google Scholar 

  13. Barger T, Alwan M, Kell S, Turner B, Wood S, Naidu A (2002) A objective remote assessment of activities of daily living: analysis of meal preparation patterns. Poster presentation, Medical Automation Research Center, University of Virginia Health System

  14. Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. In: Proceedings of CHI09 Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research

  15. Gnanavel VK, Srinivasan A (2014) Abnormal event detection in crowded video scenes. In: Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA), pp 441–448

  16. Marinho BL, Souza Junior AH, Rebouças FP (2016) A new approach to human activity recognition using machine learning techniques. In: International Conference on Intelligent Systems Design and Applications (ISDA) Intelligent Systems Design and Applications, pp 529–538

  17. Jingjing Z, Eryan Y, Xuewei C (2010) A novel frame rate conversion algorithm based on adaptive motion search. In: IEEE International Conference on Audio, Language and Image Processing

  18. Kevin H, Hyun WP (2000) Using motion-compensated frame-rate conversion for the correction of 3: 2 pulldown artifacts in video sequences. IEEE Trans Circuits Syst Video Technol 10(6):869–877

    Article  Google Scholar 

  19. Kumar V, Minz S (2014) Feature selection: a literature review. Smart Comput Rev 4(3):211–229

    Article  Google Scholar 

  20. Bobick Aaron F, Davis James W (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267

    Article  Google Scholar 

  21. Zhao T, Nevatia R (2004) Tracking multiple humans in complex situations. IEEE Trans Pattern Anal Mach Intell 26(9):1208–1221

    Article  Google Scholar 

  22. Hariataoglu I, Harwood D, Davis L (2000) W4: a real-time surveillance of people and their activities. IEEE Trans Pattern Anal Mach Intell 22(8):809–830

    Article  Google Scholar 

  23. Rmer R, Stan S (1999) 3D trajectory recovery for tracking multiple objects and trajectory guided recognition of actions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. vol 2

  24. Alexei E, Alexander B, Greg M, Jitendra M (2003) Recognizing actions at a distance. In: IEEE International Conference on Computer Vision, Nice, France

  25. Liu Y, Kender JR (2003) Sort-merge feature selection for video data. In: SDM

  26. Ribeiro P, Santos-Victor J, Lisboa P (2005) Human activity recognition from video: modeling feature selection and classification architecture. In: Proceedings of International Workshop Human Activity Recognition Model. pp 1–10

  27. Zhang M, Sawchuk AA (2011) A feature selection-based framework for human activity recognition using wearable multimodal sensors. In: Proceedings of International Conference Body Area Networks, pp 92–98

  28. Nuno CS, João MM, Paulo M (2013) Features selection for human activity recognition with iPhone inertial sensors. In: 16th Portuguese Conference on Artificial Intelligence. EPIA 2013. Angra do Heroísmo, September 9–12

  29. Hussein M, Eid E, Hoda O (2015) Evaluation of feature selection on human activity recognition. In: IEEE Seventh International Conference on Intelligent Computing and Information Systems (ICICIS)

  30. Li X, Chuah MC (2018) 2018 ReHAR: robust and efficient human activity recognition. In: IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe

  31. Dong-GyuLee Seong-WhanLee (2019) Prediction of partially observed human activity based on pre-trained deep representation. Pattern Recogn 85:198–206

    Article  Google Scholar 

  32. Wang A, Chen G, Wu X, Liu L, An N, Chang C-Y, Wang A, Chen G, Wu X, Liu L (2018) Towards human activity recognition: a hierarchical feature selection framework. Sensors 18:3629

    Article  Google Scholar 

  33. Nunesa UM, Fariab DE, Peixoto P (2017) A human activity recognition framework using max–min features and key poses with differential evolution random forests classifier. Pattern Recogn Lett 99:21–31

    Article  Google Scholar 

  34. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    Article  MathSciNet  Google Scholar 

  35. Ratrout NT, Gazder U (2014) Factors affecting performance of parametric and non-parametric models for daily traffic forecasting. Procedia Comput Sci 32:285–292

    Article  Google Scholar 

  36. Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, Cambridge

    MATH  Google Scholar 

  37. Brox T, Rosenhahn B, Cremers D, Seidel HP (2007) Non-parametric density estimation with adaptive, anisotropic kernels for human motion tracking. In: Human MotionâĂŞUnderstanding, Modeling, Capture, and Animation. Springer, Berlin, pp 152–165

  38. Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90:1151–1163

    Article  Google Scholar 

  39. Allan S, Henrik B, Sourav B (2015) Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of ACM SenSys

  40. Timo S, Heiner S (2016) On-body localization of wearable devices: an investigation of position-aware activity recognition. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 1–9

  41. Mohan S, Thirumalai C, Srivastava G (2019) Effective heart disease prediction using hybrid machine learning techniques. IEEE Access 7:81542–81554. https://doi.org/10.1109/ACCESS.2019.2923707

    Article  Google Scholar 

  42. Pawar K, Attar V (2019) Deep learning approaches for video-based anomalous activity detection. World Wide Web 22:571–601

    Article  Google Scholar 

  43. Ramachandran S, Palivela LH (2019) An intelligent system to detect human suspicious activity using deep neural networks. J Intell Fuzzy Syst 18:4507–4518

    Article  Google Scholar 

  44. Seyed YN, Yu C, Alexander A, Erik B, Timothy RF (2019) I-SAFE: instant suspicious activity identification at the edge using fuzzy decision making. arXiv

  45. Amrutha CV, Jyotsna C, Amudha J (2020) Deep learning approach for suspicious activity detection from surveillance video. In: 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bangalore, India, pp 335–339. https://doi.org/10.1109/ICIMIA48430.2020.9074920

  46. Malina L, Srivastava G, Dzurenda P, Hajny J, Fujdiak R (2019) A secure publish/subscribe protocol for internet of things. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, Aug 26, pp 1–10

  47. Dwivedi AD, Malina L, Dzurenda P, Srivastava G (2019) Optimized blockchain model for internet of things based healthcare applications. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) Jul 1, pp 135–139. IEEE

  48. Srivastava G, Lin JC, Pirouz M, Li Y, Yun U (2020) A pre-large weighted-fusion system of sensed high-utility patterns. IEEE Sen J 1:1–10

    Google Scholar 

  49. Sharma B, Srivastava G, Lin JC (2020) A bidirectional congestion control transport protocol for the internet of drones. Comput Commun 1(153):102–16

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nalla Malla Engineering College, Hyderabad, Telengana, India

    G. Vallathan

  2. School of Computer Science, Galgotias University, Greater Noida, India

    A. John

  3. School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, 632014, India

    Chandrasegar Thirumalai & SenthilKumar Mohan

  4. Department of Mathematics and Computer Science, Brandon University, Brandon, R7A 6A9, Canada

    Gautam Srivastava

  5. Research Center for Interneural Computing, China Medical University, Taichung, 40402, Taiwan, Republic of China

    Gautam Srivastava

  6. Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences, Bergen, Norway

    Jerry Chun-Wei Lin

Authors
  1. G. Vallathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. John
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chandrasegar Thirumalai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. SenthilKumar Mohan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gautam Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jerry Chun-Wei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gautam Srivastava.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vallathan, G., John, A., Thirumalai, C. et al. Suspicious activity detection using deep learning in secure assisted living IoT environments. J Supercomput 77, 3242–3260 (2021). https://doi.org/10.1007/s11227-020-03387-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-020-03387-8

Keywords

Search

Navigation