Skip to main content

Advertisement

Springer Nature Link
Log in

Enhancing Video Anomaly Detection Using Spatio-Temporal Autoencoders and Convolutional LSTM Networks

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Identifying suspicious activities or behaviors is essential in the domain of Anomaly Detection (AD). In crowded scenes, the presence of inter-object occlusions often complicates the detection of such behaviors. Therefore, developing a robust method capable of accurately detecting and locating anomalous activities within video sequences becomes crucial, especially in densely populated environments. This research initiative aims to address this challenge by proposing a novel approach focusing on AD behaviors in crowded settings. By leveraging a spatio-temporal method, the proposed approach harnesses the power of both spatial and temporal dimensions. This enables the method to effectively capture and analyze the intricate motion patterns and spatial information embedded within the continuous frames of video data. The objective is to create a comprehensive model that can efficiently detect and precisely locate anomalies within complex video sequences, specifically those featuring human crowds. The efficacy of the proposed model will be rigorously evaluated using a benchmark dataset encompassing diverse scenarios involving crowded environments. The dataset is designed to simulate real-world conditions where millions of video footage need to be continuously monitored in real time. The focus is on identifying anomalies, which might occur within short time frames, sometimes as brief as five minutes or even less. Given the challenges posed by the massive volume of data and the requirement for rapid AD, the research emphasizes the limitations of traditional Supervised Learning (SL) methods in this context.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of Data and Material

Not applicable.

Code Availability

Not Applicable.

References

  1. Ruwali A, Kumar AJS, Prakash KB, Sivavaraprasad G, Ratnam DV. Implementation of hybrid deep learning model (LSTM-CNN) for ionospheric TEC forecasting using GPS data. IEEE Geosci Remote Sens Lett. 2021;18(6):1004–8.

    Article  Google Scholar 

  2. Kantipudi MVVP, Kumar S, Jha AK. Scene text recognition based on bidirectional LSTM and deep neural network. Comput Intell Neurosci. 2021;2021:1–11.

    Article  Google Scholar 

  3. Ratnam DV, Rao KN. Bi-LSTM based deep learning method for 5G signal detection and channel estimation. AIMS Electron Electr Eng. 2021;5(4):334–41.

    Article  Google Scholar 

  4. Reddybattula KD, et al. Ionospheric TEC forecasting over an Indian low latitude location using long short-term memory (LSTM) deep learning network. Universe. 2022;8(11):562.

    Article  Google Scholar 

  5. Enireddy V, Karthikeyan C, Babu DV. OneHotEncoding and LSTM-based deep learning models for protein secondary structure prediction. Soft Comput. 2022;26(8):3825–36.

    Article  Google Scholar 

  6. Fernandes, Mannepalli K. Speech emotion recognition using deep learning LSTM for Tamil language. Pertan J Sci Technol. 2021;29(3):1915–36.

    Google Scholar 

  7. Fernandes JB, Mannepalli K. Enhanced deep hierarchal GRU & BILSTM using data augmentation and spatial features for tamil emotional speech recognition. Int J Mod Educ Comput Sci. 2022;14(3):45–63.

    Article  Google Scholar 

  8. Dharani NP, Bojja P. Analysis and prediction of COVID-19 by using recurrent LSTM neural network model in machine learning. Int J Adv Comput Sci Appl. 2022;13(5):171–8.

    Google Scholar 

  9. Divya TV, Banik BG. Detecting fake news over job posts via bi-directional long short-term memory (BIDLSTM). Int J Web-Based Learn Teach Technol. 2021;16(6):1–18.

    Article  Google Scholar 

  10. Bhimavarapu U. IRF-LSTM: enhanced regularization function in LSTM to predict the rainfall. Neural Comput Appl. 2022;34(22):20165–77.

    Article  Google Scholar 

  11. Majji R, Prakash PGO, Cristin R, Parthasarathy G. Social bat Optimisation dependent deep stacked auto-encoder for skin cancer detection. IET Image Process. 2020;14(16):4122–31.

    Article  Google Scholar 

  12. Brahmane AV, Krishna CB. Rider chaotic biography optimization-driven deep stacked auto-encoder for big data classification using spark architecture: Rider chaotic biography optimization. Int J Web Serv Res. 2021;18(3):42–62.

    Article  Google Scholar 

  13. Panneerselvam IR. Transfer learning autoencoder used for compressing multimodal biosignal. Multimedia Tools Appl. 2022;81(13):17547–65.

    Article  Google Scholar 

  14. Mahanty M, Bhattacharyya D, Midhunchakkaravarthy D. SRGAN assisted encoder-decoder deep neural network for colorectal polyp semantic segmentation. Revue d’Intelligence Artificielle. 2021;35(5):395–401.

    Article  Google Scholar 

  15. Tilak VG, Ghali VS, Kumar AD, Sankar KBS, Sharanya VSNS. Deep autoencoder for automatic defect detection in thermal wave imaging. J Green Eng. 2020;10(12):13107–18.

    Google Scholar 

  16. Kumar YP, Babu BV. Stabbing of intrusion with learning framework using auto encoder based intellectual enhanced linear support vector machine for feature dimensionality reduction. Revue d’Intelligence Artificielle. 2022;36(5):737–43.

    Article  Google Scholar 

  17. Brahmane AV, Krishna BC. DSAE-deep stack auto encoder and RCBO-rider chaotic biogeography optimization algorithm for big data classification. Adv Parallel Comput. 2021;39:213–27.

    Google Scholar 

  18. Appathurai A, Sundarasekar R, Raja C, Alex EJ, Palagan CA, Nithya A. An efficient optimal neural network-based moving vehicle detection in traffic video surveillance system. Circuits Syst Signal Process. 2020;39(2):734–56.

    Article  Google Scholar 

  19. Raju K, et al. A robust and accurate video watermarking system based on SVD hybridation for performance assessment. Int J Eng Trends Technol. 2020;68(7):19–24.

    Article  Google Scholar 

  20. Shaik AA, Mareedu VDP, Polurie VVK. Learning multiview deep features from skeletal sign language videos for recognition. Turk J Electr Eng Comput Sci. 2021;29(2):1061–76.

    Article  Google Scholar 

  21. Suneetha M, et al. Multi-view motion modelled deep attention networks (M2DA-Net) for video-based sign language recognition. J Vis Commun Image Represent. 2021;78:103161.

    Article  Google Scholar 

  22. Ghuge CA, Chandra Prakash V, Ruikar SD. Weighed query-specific distance and hybrid NARX neural network for video object retrieval. Comput J. 2020;63(7):1738–55.

    Article  Google Scholar 

  23. Mohan KK, Prasad CR, Kishore PVV. Yolo V2 with bifold skip: a deep learning model for video based real time train bogie part identification and defect detection. J Eng Sci Technol. 2021;16(3):2166–90.

    Google Scholar 

  24. Kotkar VA, Sucharita V. Scalable anomaly detection framework in video surveillance using keyframe extraction and machine learning algorithms. J Adv Res Dyn Control Syst. 2020;12(7):395–408.

    Article  Google Scholar 

  25. Suneetha M, Prasad MVD, Kishore PVV. Sharable and unshareable within class multi view deep metric latent feature learning for video-based sign language recognition. Multimedia Tools Appl. 2022;81(19):27247–73.

    Article  Google Scholar 

  26. Ali SKA, Prasad MVD, Kumar PP, Kishore PVV. Deep multi view spatio temporal spectral feature embedding on skeletal sign language videos for recognition. Int J Adv Comput Sci Appl. 2022;13(4):810–9.

    Google Scholar 

  27. Gullapelly A, Banik BG. Exploring the techniques for object detection, classification, and tracking in video surveillance for crowd analysis. Indian J Comput Sci Eng. 2020;11(4):321–6.

    Article  Google Scholar 

  28. Ghuge CA, Prakash VC, Ruikar SD. Systematic analysis and review of video object retrieval techniques. Control Cybern. 2020;49(4):471–98.

    Google Scholar 

  29. Priyadharshini B, Gomathi T. Navie bayes classifier for wireless capsule endoscopy video to detect bleeding frames. Int J Sci Technol Res. 2020;9(1):3286–91.

    Google Scholar 

  30. Ali SA, Prasad MVD, Kishore PVV. Ranked multi-view skeletal video-BASED sign language recognition with triplet loss embeddings. J Eng Sci Technol. 2022;17(6):4367–97.

    Google Scholar 

  31. Krishnamohan K, Prasad CR, Kishore PVV. Train rolling stock video segmentation and classification for bogie part inspection automation: a deep learning approach. J Eng Appl Sci. 2022. https://doi.org/10.1186/s44147-022-00128-x.

    Article  Google Scholar 

  32. Li X, Manivannan P, Anand M. Task modelling of sports event for personalized video streaming data in augmentative and alternative communication. J Interconnect Netw. 2022. https://doi.org/10.1142/S0219265921410279.

    Article  Google Scholar 

  33. Wagdarikar AMU, Senapati RK. A secure communication approach in OFDM using optimized interesting region-based video watermarking. Int J Pervasive Comput Commun. 2022;18(2):171–94.

    Article  Google Scholar 

  34. Ghuge C, Prakash VC, Ruikar S. An integrated approach using optimized naive bayes classifier and optical flow orientation for video object retrieval. Int J Intell Eng Syst. 2021;14(3):210–21.

    Google Scholar 

  35. Jadhav AD, Pellakuri V. Highly accurate and efficient two phase-intrusion detection system (TP-IDS) using distributed processing of HADOOP and machine learning techniques. J Big Data. 2021. https://doi.org/10.1186/s40537-021-00521-y.

    Article  Google Scholar 

  36. Adam A, Rivlin E, Shimshoni I, Reinitz D. Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Trans Pattern Anal Mach Intell. 2008;30(3):555–60.

    Article  Google Scholar 

  37. Hasan M, Choi J, Neumann J, Roy-Chowdhury AK, Davis LS. Learning temporal regularity in video sequences. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR). 2016. pp. 733–742.

  38. Lu C, Shi J, Jia J. Abnormal event detection at 150 fps in Matlab. In: 2013 IEEE international conference on computer vision. 2013. pp. 2720–2727.

  39. Mahadevan V, Li W, Bhalodia V, Vasconcelos N. Anomaly detection in crowded scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2010. pp. 1975–1981.

  40. Mehran R, Oyama A, Shah M. Abnormal crowd behavior detection using social force model. In: 2009 IEEE computer society conference on computer vision and pattern recognition workshops, CVPR workshops 2009. 2009. pp. 935–942.

  41. Patraucean V, Handa A, Cipolla R. Spatio-temporal video autoencoder with differentiable memory. In: International conference on learning representations, 2015. 2016. pp. 1–10.

  42. Sabokrou M, Fathy M, Hoseini M, Klette R. Real-time anomaly detection and localization in crowded scenes. In: 2015 IEEE conference on computer vision and pattern recognition workshops (CVPRW). 2015. pp. 56–62.

  43. Shi X, Chen Z, Wang H, Yeung DY, Wong W, Woo W. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Proceedings of the 28th international conference on neural information processing systems, NIPS 2015. Cambridge, MA, USA: MIT Press; 2015. pp. 802–810.

  44. Wang T, Snoussi H. Histograms of optical flow orientation for abnormal events detection. In: IEEE international workshop on performance evaluation of tracking and surveillance, PETS. 2013. pp. 45–52.

  45. Yen SH, Wang CH. Abnormal event detection using HOSF. In: 2013 International conference on IT convergence and security, ICITCS 2013. 2013.

  46. Zhao B, Fei-Fei L, Xing EP. Online detection of unusual events in videos via dynamic sparse coding. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. 2011. pp. 3313–3320

  47. Zhou S, Shen W, Zeng D, Fang M, Wei Y, Zhang Z. Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Sig Process Image Commun. 2016;47:358–68.

    Article  Google Scholar 

  48. Shakeela S, Shankar NS, Reddy PM, Tulasi TK, Koneru MM. Optimal ensemble learning based on distinctive feature selection by univariate ANOVA-F statistics for IDS. Int J Electron Telecommun. 2021;67(2):267–75.

    Google Scholar 

  49. Jadhav AD, Pellakuri V. Accuracy based fault tolerant two phase—intrusion detection system (TP-IDS) using machine learning and HDFS. Revue d’Intelligence Artificielle. 2021;35(5):359–66.

    Article  Google Scholar 

  50. Hira S, Bai A, Hira S. An automatic approach based on CNN architecture to detect Covid-19 disease from chest X-ray images. Appl Intell. 2021;51(5):2864–89.

    Article  Google Scholar 

  51. Murthy MYB, Koteswararao A, Babu MS. Adaptive fuzzy deformable fusion and optimized CNN with ensemble classification for automated brain tumor diagnosis. Biomed Eng Lett. 2022;12(1):37–58.

    Article  Google Scholar 

  52. Kumar S, Jain A, Rani S, Alshazly H, Idris SA, Bourouis S. Deep neural network based vehicle detection and classification of aerial images. Intelligent Autom Soft Comput. 2022;34(1):119–31.

    Article  Google Scholar 

  53. Lakshmi Mallika I, Venkata Ratnam D, Raman S, Sivavaraprasad G. A new ionospheric model for single frequency GNSS user applications using Klobuchar model driven by auto regressive moving average (SAKARMA) method over Indian region. IEEE Access. 2020;8:54535–53.

    Article  Google Scholar 

  54. Thirugnanasambandam K, Rajeswari M, Bhattacharyya D, Kim J-Y. Directed artificial bee colony algorithm with revamped search strategy to solve global numerical optimization problems. Autom Softw Eng. 2022. https://doi.org/10.1007/s10515-021-00306-w.

    Article  Google Scholar 

  55. Sasank VVS, Venkateswarlu S. An automatic tumour growth prediction-based segmentation using full resolution convolutional network for brain tumour. Biomed Signal Process Control. 2022;71:103090.

    Article  Google Scholar 

  56. Budati AK, Katta RB. An automated brain tumor detection and classification from MRI images using machine learning techniques with IoT. Environ Dev Sustain. 2022;24(9):10570–84.

    Article  Google Scholar 

  57. Gopi Tilak V, Ghali VS, Vijaya Lakshmi A, Suresh B, Naik RB. Proximity based automatic defect detection in quadratic frequency modulated thermal wave imaging. Infrared Phys Technol. 2021;114:103674.

    Article  Google Scholar 

  58. Bhimanpallewar RN, Narasingarao MR. AgriRobot: Implementation and evaluation of an automatic robot for seeding and fertiliser microdosing in precision agriculture. Int J Agric Resour Gov Ecol. 2020;16(1):33–50.

    Google Scholar 

  59. Thamizhazhagan P, et al. AI based traffic flow prediction model for connected and autonomous electric vehicles. Comput Mater Contin. 2022;70(2):3333–47.

    Google Scholar 

  60. Vesala GT, Ghali VS, Lakshmi AV, Naik RB. Deep and handcrafted feature fusion for automatic defect detection in quadratic frequency modulated thermal wave imaging. Russ J Nondestr Test. 2021;57(6):476–85.

    Article  Google Scholar 

  61. Vijayalakshmi A, Ghali VS, Chandrasekhar Yadav GVP, Gopitilak V, Muzammil Parvez M. Machine learning based automatic defect detection in non-stationary thermal wave imaging. ARPN J Eng Appl Sci. 2020;15(2):172–8.

    Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Networks and Cybersecurity, Faculty of Information Technology, Al Ahliyya Amman University Country, Amman, Jordan

    Ghayth Almahadin

  2. School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, 600127, Tamil Nadu, India

    Maheswari Subburaj

  3. Department of Networks and Cybersecurity, Information Technology, Al Ahliyya Amman University, Amman, Jordan

    Mohammad Hiari

  4. Department Computing Technology, School of Computing, SRM Institute of Science and Technology, Kattankulathur Campus, Chennai, 603203, Tamil Nadu, India

    Saranya Sathasivam Singaram

  5. Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur, 522302, India

    Bhanu Prakash Kolla

  6. Department of Computer Science and Engineering, Swami Keshvanand Institute of Technology, Management and Gramothan (SKIT), Jaipur, 302017, Rajasthan, India

    Pankaj Dadheech

  7. Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International (Deemed University), Pune, 411016, MH, India

    Amol D. Vibhute

  8. Department of Computer Science and Engineering, PSN College of Engineering and Technology, Tirunelveli, 627152, Tamil Nadu, India

    Sudhakar Sengan

Authors
  1. Ghayth Almahadin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maheswari Subburaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Hiari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saranya Sathasivam Singaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bhanu Prakash Kolla
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pankaj Dadheech
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Amol D. Vibhute
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sudhakar Sengan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Maheswari Subburaj or Sudhakar Sengan.

Ethics declarations

Conflict of interest

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Soft Computing in Engineering Applications” guest edited by Kanubhai K. Patel.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almahadin, G., Subburaj, M., Hiari, M. et al. Enhancing Video Anomaly Detection Using Spatio-Temporal Autoencoders and Convolutional LSTM Networks. SN COMPUT. SCI. 5, 190 (2024). https://doi.org/10.1007/s42979-023-02542-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02542-1

Keywords