Skip to main content
SpringerLink
Log in

Multiview human activity recognition using uniform rotation invariant local binary patterns

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Significant efforts have been made to monitor human activity, although it remains a challenging area for computer vision research. This paper has introduced a framework to identify the most common types of video surveillance activities. The proposed framework consists of three consecutive modules: (i) human detection by background subtraction, (ii) retrieval of uniform and rotation invariant local binary pattern (LBP) feature, and (iii) identification of human activities with a support vector machine (SVM) multiclass classifier. This framework provides a consistent view of the human actions that look at multiple subjects from different views. In addition to this, uniform patterns provide better performance in discriminating human activities. A multiclass SVM is used for classification of human activities. SVM classifier is set and trained to achieve the better efficiency by selecting the appropriate feature before it is integrated. Weizmann's Multiview dataset, CASIA dataset and IXMAS dataset confirm the high efficiency and better robustness of the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv 43(3):1–43

    Article  Google Scholar 

  • Ahonen T, Matas J, He C, Pietikäinen M (2009). Rotation invariant image description with local binary pattern histogram fourier features. In: Scandinavian conference on image analysis. Springer, Berlin, Heidelberg, pp 61–70

  • Bianconi F, Fernández A (2011) On the occurrence probability of local binary patterns: a theoretical study. J Mathematical Imaging Vis 40(3):259–268

    Article  MathSciNet  MATH  Google Scholar 

  • Binh NT, Nigam S, Khare A (2013) Towards classification based human activity recognition in video sequences. International Conference on Context-Aware Systems and Applications. Springer, Cham, pp 209–218

    Google Scholar 

  • Cheng Z, Qin L, Huang Q, Yan S, Tian Q (2014) Recognizing human group action by layered model with multiple cues. Neurocomputing 136:124–135

    Article  Google Scholar 

  • Fernández A, Ghita O, González E, Bianconi F, Whelan PF (2011) Evaluation of robustness against rotation of LBP, CCR and ILBP features in granite texture classification. Mach Vis Appl 22(6):913–926

    Article  Google Scholar 

  • Gorelick L, Blank M, Shechtman E, Irani M, Basri R (2007) Actions as space-time shapes. IEEE Trans Pattern Anal Mach Intell 29(12):2247–2253

    Article  Google Scholar 

  • Iosifidis A, Nikolaidis N, Pitas I (2010) Movement recognition exploiting multi-view information. In: 2010 IEEE International Workshop on Multimedia Signal Processing, IEEE, pp 427–431

  • Jan A, Khan GM (2021) Real world anomalous scene detection and classification using multilayer deep neural networks. Int J Interactive Multimedia Artif Intell. https://doi.org/10.9781/ijimai.2021.10.010

  • Ji X, Liu H (2009) Advances in view-invariant human motion analysis: a review. IEEE Trans Syst Man Cybern Part C (Appl Rev) 40(1):13–24

    Google Scholar 

  • Ji Y, Yang Y, Shen HT, Harada T (2021) View-invariant action recognition via Unsupervised AttentioN Transfer (UANT). Pattern Recogn 113:107807

    Article  Google Scholar 

  • Kellokumpu V, Zhao G, Pietikäinen M (2011) Recognition of human actions using texture descriptors. Mach Vis Appl 22(5):767–780

    Article  Google Scholar 

  • Kellokumpu V, Zhao G, Pietikäinen M (2010) Dynamic textures for human movement recognition. In: Proceedings of the ACM International Conference on Image and Video Retrieval, pp 470–476

  • Lahdenoja O, Poikonen J, Laiho M (2013) Towards understanding the formation of uniform local binary patterns. International Scholarly Research Notices

  • Lan T, Wang Y, Yang W, Robinovitch SN, Mori G (2011) Discriminative latent models for recognizing contextual group activities. IEEE Trans Pattern Anal Mach Intell 34(8):1549–1562

    Article  Google Scholar 

  • Li Y, Xu X, Xu J, Du E (2019) Bilayer model for cross-view human action recognition based on transfer learning. J Electron Imaging 28(3):033016

    Article  Google Scholar 

  • Lv Z, Qiao L, Singh AK, Wang Q (2021a) AI-empowered IoT security for smart cities. ACM Trans Internet Technol 21(4):1–21

    Google Scholar 

  • Lv Z, Qiao L, Singh AK, Wang Q (2021b) Fine-grained visual computing based on deep learning. ACM Trans Multimedia Comput Commun Appl 17(1s):1–19

    Article  Google Scholar 

  • Lv Z, Guo J, Singh AK, Lv H (2022) Digital twins based VR simulation for accident prevention of intelligent vehicle. IEEE Trans Vehicular Tech 71(4):3414–3428

    Article  Google Scholar 

  • Määttä T, Härmä A, Aghajan H (2010) On efficient use of multi-view data for activity recognition. In: Proceedings of the Fourth ACM/IEEE International Conference on Distributed Smart Cameras, pp 158–165

  • Matikainen P, Pillai P, Mummert L, Sukthankar R, Hebert M (2011) Prop-free pointing detection in dynamic cluttered environments. In 2011 IEEE International Conference on Automatic Face and Gesture Recognition (FG), IEEE, pp 374–381

  • Nigam S, Khare A (2016) Integration of moment invariants and uniform local binary patterns for human activity recognition in video sequences. Multimedia Tools Appl 75(24):17303–17332

    Article  Google Scholar 

  • Nigam S, Singh R, Misra AK (2018) Efficient facial expression recognition using histogram of oriented gradients in wavelet domain. Multimedia Tools Appl 77(21):28725–28747

    Article  Google Scholar 

  • Nigam S, Singh R, Misra AK (2019) A review of computational approaches for human behavior detection. Arch Comput Methods Eng 26(4):831–863

    Google Scholar 

  • Nigam S, Singh R, Singh MK, Singh VK (2021) Multiple views based recognition of human activities using uniform patterns. In: 2021 Sixth International Conference on Image Information Processing (ICIIP), IEEE vol. 6, pp 483–488

  • Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  MATH  Google Scholar 

  • Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns, vol 40. Springer Science and Business Media

    Google Scholar 

  • Pillai MS, Chaudhary G, Khari M, Crespo RG (2021) Real-time image enhancement for an automatic automobile accident detection through CCTV using deep learning. Soft Comput 25(18):11929–11940

    Article  Google Scholar 

  • Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28(6):976–990

    Article  Google Scholar 

  • Rajagopal A, Joshi GP, Ramachandran A, Subhalakshmi RT, Khari M, Jha S, You J (2020) A deep learning model based on multi-objective particle swarm optimization for scene classification in unmanned aerial vehicles. IEEE Access 8:135383–135393

    Article  Google Scholar 

  • Saha A, Rajak S, Saha J, Chowdhury C (2022) A survey of machine learning and meta-heuristics approaches for sensor-based human activity recognition systems. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-022-03870-5

    Article  Google Scholar 

  • Sahoo KS, Tripathy BK, Naik K, Ramasubbareddy S, Balusamy B, Khari M, Burgos D (2020) An evolutionary SVM model for DDOS attack detection in software defined networks. IEEE Access 8:132502–132513

    Article  Google Scholar 

  • Singh R, Nigam S, Singh AK, Elhoseny M (2020a) Intelligent wavelet based techniques for advanced multimedia applications. Springer, pp 1–144

    Google Scholar 

  • Singh R, Ahmed T, Kumar A, Singh AK, Pandey AK, Singh SK (2020b) Imbalanced breast cancer classification using transfer learning. IEEE/ACM Trans Comput Biol Bioinf 18(1):83–93

    Google Scholar 

  • Souvenir R, Babbs J (2008) Learning the viewpoint manifold for action recognition. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp 1–7

  • Srivastava S, Khari M, Crespo RG, Chaudhary G, Arora P (eds) (2021) Concepts and real-time applications of deep learning. Springer International Publishing

    Google Scholar 

  • Verma KK, Singh BM (2021) Deep multi-model fusion for human activity recognition using evolutionary algorithms. Int J Interact Multimedia Artif Intell 7(2):44

    Google Scholar 

  • Verma KK, Singh BM, Mandoria HL, Chauhan P (2020) Two-stage human activity recognition using 2DConvNet. Int J Interactive Multimedia Artif Intell. https://doi.org/10.9781/ijimai.2020.04.002

    Article  Google Scholar 

  • Vili K, Guoying Z, Matti P (2008) Texture based description of movements for activity analysis. In International Conference on Computer Vision Theory and Applications (VISAPP 2008), vol 1, pp 206–213

  • Vrigkas M, Karavasilis V, Nikou C, Kakadiaris IA (2014) Matching mixtures of curves for human action recognition. Comput Vis Image Underst 119:27–40

    Article  Google Scholar 

  • Vyas S, Rawat YS, Shah M (2020) Multi-view action recognition using cross-view video prediction. In European Conference on Computer Vision. Springer, Cham, pp 427–444

  • Wang Y, Mori G (2010) Hidden part models for human action recognition: Probabilistic versus max margin. IEEE Trans Pattern Anal Mach Intell 33(7):1310–1323

    Article  Google Scholar 

  • Wang Y, Xiao Y, Lu J, Tan B, Cao Z, Zhang Z, Zhou JT (2021) Discriminative multi-view dynamic image fusion for cross-View 3-D action recognition. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3070179

    Article  Google Scholar 

  • Wang Y, Huang K, Tan T (2007) Human activity recognition based on r transform. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp 1–8

  • Weinland D, Ronfard R, Boyer E (2006) Free viewpoint action recognition using motion history volumes. Comput Vis Image Underst 104(2–3):249–257

    Article  Google Scholar 

  • Weinland D, Ronfard R, Boyer E (2011) A survey of vision-based methods for action representation, segmentation and recognition. Comput Vis Image Underst 115(2):224–241

    Article  Google Scholar 

  • Weinland D, Özuysal M, Fua P (2010) Making action recognition robust to occlusions and viewpoint changes. In: European Conference on Computer Vision. Springer, Berlin, Heidelberg, pp 635–648

  • Wu J, Hu D, Chen F (2014) Action recognition by hidden temporal models. Vis Comput 30(12):1395–1404

    Article  Google Scholar 

  • Yousef R, Gupta G, Yousef N, Khari M (2022) A holistic overview of deep learning approach in medical imaging. Multimedia Syst 28(3):881–914

    Article  Google Scholar 

  • Zhao G, Ahonen T, Matas J, Pietikainen M (2011) Rotation-invariant image and video description with local binary pattern features. IEEE Trans Image Process 21(4):1465–1477

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

“This work was supported in part by the Ministry of Electronics and Information Technology (MeitY), Government of India under Grant No. 3(9)/2021-EG-II.”

Author information

Authors and Affiliations

  1. Department of Computer Science, Banasthali Vidyapith, Banasthali, India

    Swati Nigam & Rajiv Singh

  2. Department of Computer Science, Banaras Hindu University, Varanasi, India

    Manoj Kumar Singh & Vivek Kumar Singh

Authors
  1. Swati Nigam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajiv Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manoj Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vivek Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajiv Singh.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest associated with the manuscript and data associated will be provided on a reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nigam, S., Singh, R., Singh, M.K. et al. Multiview human activity recognition using uniform rotation invariant local binary patterns. J Ambient Intell Human Comput 14, 4707–4725 (2023). https://doi.org/10.1007/s12652-022-04374-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-04374-y

Keywords

Search

Navigation