Skip to main content
SpringerLink
Log in

Action recognition based on binary patterns of action-history and histogram of oriented gradient

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

In this paper, we have focused on the view-based spatio-temporal template matching approach for human action detection and classification. We have proposed an approach for human activity modeling that describes human motions as a texture pattern. We have combined two relatively simple feature extractors for obtaining a system to get more accurate result. In this method, video sequences are converted into temporal templates called Motion History Image (MHI), which can preserve dominant motion information. The local features are described with Local Binary Pattern (LBP) and Histogram of Oriented Gradients (HOG) descriptors. LBP operator is texture operator that encodes the direction of motion from the non-monotonous areas of MHI images. HOG is used as feature descriptor and extracts the features from LBP. These descriptors are used to train with Support Vector Machine (SVM) classifier to recognize various action classes. This proposed method has been tested on the KTH Action Dataset (which is one of the most widely used benchmark datasets for human action classification), and on the Pedestrian Action Dataset. Our method has shown 86.67 % recognition rate in the 6-classes of KTH Action Dataset and 94.3 % accuracy in the 7-classes of Pedestrian Action Dataset. Based on the complexity of datasets, both the results are quite satisfactory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Ahad MAR (2011) “Computer vision and action recognition: a guide for image processing and computer vision community for action understanding”, 1st edn. Atlantis Ambient and Pervasive Intelligence and Springer, Berlin

  2. Ahad MAR (2013) “Motion history images for action recognition and understanding”, 1st edn. Springer, Berlin

  3. Ahad MAR, Tan JK, Kim H, Ishikawa S (2008) “Human activity recognition: various paradigms”. International Conference in Control, Automation and Systems, pp 1896–1901

  4. Schuldt C, Laptev I, Caputo B (2004) “Recognizing human actions: a local SVM approach”. International Conference on Pattern Recognition, pp 32–36

  5. Mueid RM, Ahmed C, Ahad MAR (2015) “Pedestrian activity classification using patterns of motion and histogram of oriented gradient”. J Multimodal User Interfaces 1–7. doi:10.1007/s12193-015-0178-3

  6. Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488

    Article  Google Scholar 

  7. Moeslund TB, Granum E (2001) A survey of computer vision-based human motion capture. Comput Vis Image Understand 106:90–126

    MATH  Google Scholar 

  8. Wang L, Hu W, Tan T (2003) Recent developments in human motion analysis. Pattern Recognit 36:585–601

    Article  Google Scholar 

  9. Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Understand 108:4–18

    Article  Google Scholar 

  10. Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28:976–990

    Article  Google Scholar 

  11. Moeslund TB, Hilton A, Kruger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Understand 104(2–3):90–126

    Article  Google Scholar 

  12. Gavrila DM (1999) “The visual analysis of human movement: a survey”. Comput Vis Image Understand 73:82–98

  13. Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Understand 73(3):428–440

    Article  Google Scholar 

  14. Bobick A, Davis J (1996) “An Appearance-based Representation of Action”. International Conference on Pattern Recognition, pp 307–312

  15. Davis JW (1996) “Appearance-based motion recognition of human actions”, M.I.T. Media Lab Perceptual Computing Group Tech. Report No. 387, p 51

  16. Anderson C, Bert P, Wal GV (1985) Change detection and tracking using pyramids transformation techniques. SPIE-Intell Robot Comput Vis 579:72–78

    Google Scholar 

  17. Lipton AJ, Fujiyoshi H, Patil RS (1998) “Moving Target classification and tracking from real-time video”. IEEE Workshop on Applications of Computer Vision, pp 8–14

  18. Ahad MAR, Tan JK, Kim H, Ishikawa S (2009) Temporal motion recognition and segmentation approach. Int J Imaging Syst Technol 19:91–99

    Article  Google Scholar 

  19. Ahad MAR, Ogata T, Tan JK, Kim H, Ishikawa S (2008) A complex motion recognition technique employing directional motion templates. Int J Innov Comput, Inf Control 4(8):1943–1954

    Google Scholar 

  20. Gavrila DM (2000) Pedestrian detection form a moving vehicle, vol 1843. Springer, Berlin. pp 37–49

  21. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans on PAMI 27(10):1615–1630

    Article  Google Scholar 

  22. Dalal N, Triggs B (2005) “Histograms of oriented gradients for human detection”. International Conference on Computer Vision and Pattern Recognition, pp 886–893

  23. Bobick A, Davis J (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267

    Article  Google Scholar 

  24. Meng H, Pears N, Bailey C (2007) “A human action recognition system for embedded computer vision application”. In: Workshop on Embedded Computer Vision (with CVPR), pp 1–6

  25. Han J, Bhanu B (2006) Individual recognition using gait energy image. IEEE Trans PAMI 28(2):316–322

    Article  Google Scholar 

  26. Chandrashekhar V, Venkatesh KS (2006) “Action energy images for reliable human action recognition”. In: Asian Symposium on Information Display (ASID), pp 484–487

  27. Liu J, Zhang N (2007) “Gait history image: a novel temporal template for gait recognition”. IEEE International Conference on Multimedia and Expo, pp 663–666

  28. Ahad MAR, Tan J, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281

    Article  Google Scholar 

  29. Ojala T, Pietikainen M, Harwood D (1994) Performance evaluation of texture measures with classification based on kullback discrimination of distributions. Int Conf Pattern Recognit 1:582–585

    Article  Google Scholar 

  30. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recognit 29(1):51–59

    Article  Google Scholar 

  31. Pietikainen M, Zhao G, Hadid A, Ahonen T (2011) “Computer vision using local binary pattern”, 1st edn. Springer, Berlin

  32. Kellokumpu V, Zhao G, Pietikainen M (2008) Texture based description of movements for activity analysis. Int Conf Comput Vis Theory Appl 1:206–213

    Google Scholar 

  33. Heikkila J, Pietikainen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans PAMI 28(4):657–662

    Article  Google Scholar 

  34. Heikkila J, Pietikainen M (2004) “A texture-based method for detecting moving objects”. British Machine Vision Conference, pp 187–196

  35. Hadid A, Pietikainen M (2009) Combining appearance and motion for face and gender recognition from videos. Pattern Recognit 42(11):2818–2827

    Article  Google Scholar 

  36. Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans PAMI 29(6):915–928

    Article  Google Scholar 

  37. Huang C, Hsieh C, Lai K, Huang WY (2011) “Human action recognition using histogram of oriented gradient of motion history image”. In: International conference on instrumentation, measurement, computer, communication and control, pp 353–356

  38. Bertozzi M, Broggi A, Del Rose M, Felisa M, Rakotomamonjy A, Suard F (2007) “A pedestrian detector using histograms of oriented gradients and a support vector machine classifier”. In: IEEE intelligent transportation systems conference, pp 143–144

  39. Dalal N, Triggs B, Schmid C (2006) “Human detection using oriented histograms of flow and appearance”. In: European Conference on Computer Vision, pp 428–441

  40. Osuna E, Freund R, Girosi F (1997) “Training support vector machines: an application to face detection”. CVPR, pp 130–136

  41. Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans neural Netw 10(5):988–999

    Article  Google Scholar 

  42. Weinland D, Ronfard R, Boyer E (2006) Free viewpoint action recognition using motion history volumes. Comput Vis Image Understand 104:249–257

    Article  Google Scholar 

  43. Wolf L, Shashua A (2003) “Kernel principal angles for classification machines with applications to image sequence interpretation”. CVPR, pp 635–640

  44. Wallraven C, Caputo B, Graf A (2003) “Recognition with local features: the kernel recipe”. ICCV, pp 257–264

  45. Klaser A, Marszalek M, Schmid C (2008) “A spatio-temporal descriptor based on 3D gradients”. British machine vision conference

  46. Gilbert A, Illingworth J, Bowden R (2011) Action recognition using mined hierarchical compound features. IEEE Trans Pattern Anal Mach Intell 33(5): 883–897. doi:10.1109/TPAMI.2010.144

  47. Niebles J, Wang H, Fei-Fei L (2006) “Unsupervised learning of human action categories using spatial-temporal words”. In: British machine vision conference

  48. Dollar P, Rabaud V, Cottrell G, Belongie S (2005) “Behavior recognition via sparse spatiotemporal features”. In: International workshop on visual surveillance and performance evaluation of tracking and surveillance, pp 65–72

  49. Wong S, Cipolla R, “Extracting spatio-temporal interest points using global information”. In: International conference on computer vision

  50. Bregonzio M, Li J, Gong S, Xiang T (2010) “Discriminative topics modeling for action feature selection and recognition”. In: British machine vision conference

Download references

Acknowledgments

Authors are thankful to the Center for Natural Science and Engineering Research (CNSER) and UGC, Bangladesh.

Author information

Authors and Affiliations

  1. University of Dhaka, Dhaka, Bangladesh

    Md. Atiqur Rahman Ahad

  2. Department of Electrical and Electronic Engineering, University of Dhaka, Dhaka, 1000, Bangladesh

    Md. Nazmul Islam & Israt Jahan

Authors
  1. Md. Atiqur Rahman Ahad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Md. Nazmul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Israt Jahan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Atiqur Rahman Ahad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahman Ahad, M., Islam, M.N. & Jahan, I. Action recognition based on binary patterns of action-history and histogram of oriented gradient. J Multimodal User Interfaces 10, 335–344 (2016). https://doi.org/10.1007/s12193-016-0229-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-016-0229-4

Keywords

Search

Navigation