Skip to main content
SpringerLink
Log in

Integration of moment invariants and uniform local binary patterns for human activity recognition in video sequences

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this study, we present a method for human activity recognition in video sequences. Human activities are often described by a holistic feature vector comprising of a set of local motion descriptors. Here, we use a novel local shape feature descriptor for human activity recognition which is an integration of moment invariants and uniform local binary patterns (MI_ULBP). This feature descriptor is passed to a binary support vector machine pattern classifier for classification of human activities. Activity recognition is achieved through probabilistic search of image feature database representing previously seen activities. Experiments are performed over four benchmark video datasets Weizmann, KTH, CASIA and Collective human activity. Visual results and quantitative comparisons with existing methods show that the proposed method gives better recognition of human activities in video sequences with varying backgrounds and viewpoints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv (CSUR) 43(3): 16. Available online http://cvrc.ece.utexas.edu/aggarwaljk/Publications/review_ryoo_hdr.pdf

  2. Baumann A, Boltz M, Ebling J, Koenig M, Loos HS, Merkel M, Niem W, Warzelhan JK, Yu J (2008) A review and comparison of measures for automatic video surveillance systems. EURASIP J Image Video Process 2008(824726):1–30

    Article  Google Scholar 

  3. Binh NT, Nigam S, Khare A (2014) Towards classification based human activity recognition in video sequences. In: International conference on context aware systems and applications (ICCASA’14). Springer International Publishing, pp 209–218

  4. Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as space-time shapes. In: Tenth IEEE International Conference on Computer Vision (ICCV 2005), (Vol. 2, pp. 1395–1402)

  5. Borges PVK, Conci N, Cavallaro A (2013) Video-based human behavior understanding: a survey. IEEE Trans Circuits Syst Video Technol 23(11):1993–2008

    Article  Google Scholar 

  6. Cao H, Nguyen MN, Phua C, Krishnaswamy S, Li X (2012) An integrated framework for human activity classification. In: ACM Conference on Ubiquitous Computing (UbiComp’12), pp. 331–340

  7. Chan M, Estève D, Escriba C, Campo E (2008) A review of smart homes—present state and future challenges. Comput Methods Prog Biomed 91(1):55–81

    Article  Google Scholar 

  8. Chen BJ, Shu HZ, Zhang H, Chen G, Toumoulin C, Dillenseger JL, Luo LM (2012) Quaternion Zernike moments and their invariants for color image analysis and object recognition. Signal Process 92(2):308–318

    Article  Google Scholar 

  9. Cheng Z, Qin L, Huang Q, Yan S, Tian Q (2014) Recognizing human group action by layered model with multiple cues. Neurocomputing 136:124–135

    Article  Google Scholar 

  10. Choi W, Shahid K, Savarese S (2009) What are they doing?: Collective activity classification using spatio-temporal relationship among people. In: 12th IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1282–1289

  11. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  12. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 I.E. Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), (vol. 1, pp. 886–893)

  13. Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874

    Article  MathSciNet  Google Scholar 

  14. Fletcher T (2009) Support vector machines explained. URL: http://www.tristanfletcher.co.uk/SVM%20Explained.pdf

  15. Flusser J, Zitova B, Suk T (2009) Moments and moment invariants in pattern recognition. Wiley

  16. Garibotto G, Murrieri P, Capra A, De Muro S, Petillo U, Flammini F, Esposito M, Pragloila C, Di Leo G, Lengu R, Mazzino N, Paolillo A, D’Urso M, Vertucci R, Narducci F, Ricciardi S, Casanova A, Fenu G, De Mizio M, Savastano M, Di Capua M, Ferone A (2013) White paper on industrial applications of computer vision and pattern recognition. In: Image analysis and processing (ICIAP’13). Springer Berlin Heidelberg, pp 721–730

  17. Gonzàlez J, Moeslund TB, Wang L (2012) Semantic understanding of human behaviors in image sequences: from video-surveillance to video-hermeneutics. Comput Vis Image Underst 116(3):305–306

    Article  Google Scholar 

  18. Hosny KM (2010) Refined translation and scale Legendre moment invariants. Pattern Recogn Lett 31(7):533–538

    Article  Google Scholar 

  19. Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8(2):179–187

    Article  MATH  Google Scholar 

  20. Ikizler-Cinbis N, Sclaroff S (2010) Object, scene and actions: combining multiple features for human action recognition. In: European Conference on Computer Vision (ECCV’10). Springer Berlin Heidelberg, pp 494–507

  21. Junejo IN, Dexter E, Laptev I, Perez P (2011) View-independent action recognition from temporal self-similarities. IEEE Trans Pattern Anal Mach Intell 33(1):172–185

    Article  Google Scholar 

  22. Kellokumpu V, Zhao G, Pietikäinen M (2010) Dynamic textures for human movement recognition. In: ACM International Conference on Image and Video Retrieval (pp. 470–476)

  23. Kellokumpu V, Zhao G, Pietikäinen M (2011) Recognition of human actions using texture descriptors. Mach Vis Appl 22(5):767–780

    Article  Google Scholar 

  24. Ko BC, Kim DY, Jung JH, Nam JY (2013) Three-level cascade of random forests for rapid human detection. Opt Eng 52(2):027204–027204

    Article  Google Scholar 

  25. Lahdenoja O, Poikonen J, Laiho M (2013) Towards understanding the formation of uniform local binary patterns. International Scholarly Research Notices

  26. Lane ND, Lin M, Mohammod M, Yang X, Lu H, Cardone G, Ali S, Doryab A, Berke E, Campbell AT, Choudhury T (2014) BeWell: sensing sleep, physical activities and social interactions to promote wellbeing. Mob Netw Appl 19(3):345–359

    Article  Google Scholar 

  27. Lee L, Romano R, Stein G (2000) Introduction to the special section on video surveillance. IEEE Trans Pattern Anal Mach Intell 22(8):745

    Article  Google Scholar 

  28. Lipton A, Kanade T, Fujiyoshi H, Duggins D, Tsin Y, Tolliver D, Enomoto N, Hasegawa O, Burt P, Wixson L (2000) A system for video surveillance and monitoring, vol 2. Carnegie Mellon University, the Robotics Institute, Pittsburg

    Google Scholar 

  29. Liu Y, Jia LK, Yu WY (2014) Capturing human motion based on modified hidden markov model in multi-view image sequences. J Multimed 9(1):92–98

    Google Scholar 

  30. Liu J, Shah M, Kuipers B, Savarese S (2011) Cross-view action recognition via view knowledge transfer. In: 2011 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 3209–3216)

  31. Loy CC (2010) Activity understanding and unusual event detection in surveillance videos (Doctoral dissertation). Queen Mary University of London

  32. Mäenpää T (2003) The local binary pattern approach to texture analysis: extensions and applications. (Doctoral dissertation). University of Oulu

  33. Mäenpää T, Pietikäinen M (2005) Texture analysis with local binary patterns. Handb Pattern Recognit Comput Vis 3:197–216

    Article  MATH  Google Scholar 

  34. Mattivi R, Shao L (2010) Spatio-temporal dynamic texture descriptors for human motion recognition. In: Intelligent Video Event Analysis and Understanding. Springer Berlin Heidelberg, pp 69–91

  35. Mu Y, Yan S, Liu Y, Huang T, Zhou B (2008) Discriminative local binary patterns for human detection in personal album. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08), (pp. 1–8)

  36. Nguyen DT, Ogunbona PO, Li W (2013) A novel shape-based non-redundant local binary pattern descriptor for object detection. Pattern Recogn 46(5):1485–1500

    Article  Google Scholar 

  37. Nigam S, Deb K, Khare A (2013) Moment invariants based object recognition for different pose and appearances in real scenes. In: 2013 IEEE International Conference on Informatics, Electronics and Vision (ICIEV’13), (pp. 1–5)

  38. Nigam S, Khare A (2015) Multi-resolution approach for multiple human detection using moments and local binary patterns, Multimed Tools Appl 74(17):7037--7062

  39. Nigam S, Khare M, Srivastava RK, Khare A (2013) An effective local feature descriptor for object detection in real scenes. In: 2013 IEEE Conference on Information and Communication Technologies (ICT’13), (pp. 244–248)

  40. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  MATH  Google Scholar 

  41. Olaru A, Florea AM, Seghrouchni AEF (2013) A context-aware multi-agent system as a middleware for ambient intelligence. Mob Netw Appl 18(3):429–443

    Article  Google Scholar 

  42. Pang Y, Yuan Y, Li X, Pan J (2011) Efficient HOG human detection. Signal Process 91(4):773–781

    Article  MATH  Google Scholar 

  43. Pehlivan S, Forsyth DA (2014) Multiple view activity recognition without reconstruction. Image Vision Comput, Article in press.

  44. Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns (vol. 40). Springer

  45. Powers DM (2011) Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2(1):37–63

    MathSciNet  Google Scholar 

  46. Qian H, Mao Y, Xiang W, Wang Z (2010) Recognition of human activities using SVM multi-class classifier. Pattern Recogn Lett 31(2):100–111

    Article  Google Scholar 

  47. Sacchi C, Regazzoni CS (2000) A distributed surveillance system for detection of abandoned objects in unmanned railway environments. IEEE Trans Veh Technol 49(5):2013–2026

    Article  Google Scholar 

  48. Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04) (vol. 3, pp. 32–36)

  49. Shen J, Yang W, Sun C (2013) Real-time human detection based on gentle MILBoost with variable granularity HOG-CSLBP. Neural Comput & Applic 23(7–8):1937–1948

    Article  Google Scholar 

  50. Skibbe H, Reisert M, Schmidt T, Brox T, Ronneberger O, Burkhardt H (2012) Fast rotation invariant 3D feature computation utilizing efficient local neighborhood operators. IEEE Trans Pattern Anal Mach Intell 34(8):1563–1575

    Article  Google Scholar 

  51. Suk T, Flusser J (2003) Combined blur and affine moment invariants and their use in pattern recognition. Pattern Recogn 36(12):2895–2907

    Article  MATH  Google Scholar 

  52. Vishwakarma S, Agrawal A (2013) A survey on activity recognition and behavior understanding in video surveillance. Vis Comput 29(10):983–1009

    Article  Google Scholar 

  53. Wang Y, Huang K, Tan T (2007) Human activity recognition based on R transform. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), (pp. 1–8)

  54. Weinland D, Ronfard R, Boyer E (2006) Free viewpoint action recognition using motion history volumes. Comput Vis Image Underst 104(2):249–257

    Article  Google Scholar 

  55. Wu J, Hu D, Chen F (2013) Action recognition by hidden temporal models. Vis Comput 30(12):1395–1404

    Article  Google Scholar 

  56. Yussiff AL, Yong SP, Baharudin BB (2014) Detecting people using histogram of oriented gradients: a step towards abnormal human activity detection. In: Advanced in Computer Science and its Applications (pp. 1145–1150). Springer Berlin Heidelberg

  57. Zhao Y, Belkasim S (2012) Multiresolution Fourier descriptors for multiresolution shape analysis. IEEE Signal Process Lett 19(10):692–695

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Council of Scientific and Industrial Research, Human Resource Development Group, India via grant number 09/001/(0362)/2012/EMR-I.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication, University of Allahabad, Allahabad, 211002, India

    Swati Nigam & Ashish Khare

Authors
  1. Swati Nigam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashish Khare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashish Khare.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nigam, S., Khare, A. Integration of moment invariants and uniform local binary patterns for human activity recognition in video sequences. Multimed Tools Appl 75, 17303–17332 (2016). https://doi.org/10.1007/s11042-015-3000-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3000-z

Keywords

Search

Navigation