Skip to main content
Springer Nature Link
Log in

Human action recognition on depth dataset

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Human action recognition is a hot research topic; however, the change in shapes, the high variability of appearances, dynamitic background, potential occlusions in different actions and the image limit of 2D sensor make it more difficult. To solve these problems, we pay more attention to the depth channel and the fusion of different features. Thus, we firstly extract different features for depth image sequence, and then, multi-feature mapping and dictionary learning model (MMDLM) is proposed to deeply discover the relationship between these different features, where two dictionaries and a feature mapping function are simultaneously learned. What is more, these dictionaries can fully characterize the structure information of different features, while the feature mapping function is a regularization term, which can reveal the intrinsic relationship between these two features. Large-scale experiments on two public depth datasets, MSRAction3D and DHA, show that the performances of these different depth features have a big difference, but they are complementary. Further, the features fusion by MMDLM is very efficient and effective on both datasets, which is comparable to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://research.microsoft.com/enus/um/people/zliu/actionrecorsrc/default.htm.

  2. http://mclab.citi.sinica.edu.tw/dataset/dha/dha.html.

References

  1. Lin Y-C, Hua M-C, Cheng W-H, Hsieh Y-H, Chen H-M (2012) Human action recognition and retrieval using sole depth information, ACM MM 2012, pp 1–8

  2. Wang J, Liu Z, Wu Y, Yuan J (2012) Mining actionlet ensemble for action recognition with depth cameras. CPRR, pp 1290–1297

  3. Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. Human Communicative Behavior Analysis Workshop (in conjunction with CVPR), 2010, pp 2, 5, 6

  4. Ni B, Wang G, Moulin P (2012) RGBD-HuDaAct: A color-depth video database for human daily activity recognition. ICCV workshop, pp 1–8

  5. Megavannan V, Agarwal B, Venkatesh Babu R (2012) Human action recognition using depth maps. In: International conference on signal processing and communications (SPCOM), pp 1–8

  6. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE TPAMI 31(2):210–227

    Article  Google Scholar 

  7. Wang JJY, Bensmail H, Yao N, Gao X (2013) Discriminative sparse coding on multi-manifolds. Knowl Based Syst 54:199–206

    Article  Google Scholar 

  8. Wang JJY, Bensmail H, Gao X (2013) Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification. Pattern Recognit 46(12):3249–3255

    Article  Google Scholar 

  9. Uddin MD, Thang ND, Kim JT, Kim T-S (2011) Human activity recognition using body joint-angle features and hidden markov model. ETRI J 33(4):569–579

    Article  Google Scholar 

  10. Jalal A, Kim JT, Kim T-S (2012) Human activity recognition using the labeled depth body parts information of depth silhouettes. In: Proceeding of the 6th international symposium on sustainable healthy buildings

  11. Hu M-C, Chen C-W, Cheng W-H, Chang C-H et al (2014) Real-time human movement retrieval and assessment with Kinect sensor. IEEE Trans Cybern 45(4). doi:10.1109/TCYB.2014.2335540

  12. Ofli F, Chaudhry R, Kurillo G et al (2012) Sequence of the most informative joints (SMIJ): a new representation for human skeletal action recognition. In: Proceeding of IEEE conference on CVPR workshop, pp 8–13

  13. Xia L, Chen C-C, Aggarwal JK (2012) View invariant human action recognition using histograms of 3D joints. In: Proceeding of IEEE conference on CVPR workshop, pp 20–27

  14. Gao Z, Zhang H, Liu A-A, Xue Y-b, Guang-ping X (2014) Human action recognition using pyramid histograms of oriented gradients and collaborative multi-task learning. KSII Trans Internet Inf Syst 8(2):483–503

    Article  Google Scholar 

  15. Schwarz LA, Mateus D, Castaneda V, Navab N (2010) Manifold learning for TOF-based human body tracking and activity recognition. In: Proceeding of the British machine vision conference, pp 1–11

  16. Yang X, Zhang C, Tian Y (2012) Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceeding of ACM multimedia, pp 1057–1060

  17. Gao Z, Song J, Zhang H, Liu AA, Xue Y, Xu G (2014) Human action recognition via multi-modality information. J Electr Eng Technol 9(2):739–748

    Article  Google Scholar 

  18. Wang J, Liu Z, Chorowski J, Chen Z, Ying W (2012) Robust 3D action recognizing with random occupancy patterns. Proc ECCV 2:872–885

    Google Scholar 

  19. Vieira AW, Nascimento ER, Oliveira GL et al (2012) STOP: space-time occupancy patterns for 3D action recognition from depth map sequences. In: Proceeding of 17th Iberoamerican congress on pattern recognition, pp 252–259

  20. Gao Z, Chen M, Hauptmann AG, Cai A (2010) Comparing evaluation protocols on the KTH dataset. In: International conference on pattern recognition, pp 88–100

  21. Xia L, Aggarwal JK (2013) Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: 24th IEEE conference on computer vision and pattern recognition (CVPR), Portland, Oregon, June 2013

  22. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    Article  MATH  Google Scholar 

  23. Gao Y, Wang M, Ji R, Wu X, Dai Q (2014) 3D object retrieval with hausdorff distance learning. IEEE Trans Ind Electron 61(4):2088–2098

    Article  Google Scholar 

  24. Hu M-C, Cheng W-H, Hu C-S et al (2014) Efficient human detection in crowded environment. Multimedia Systems, pp 1432–1882

  25. Lin D, Tang X (2005) Coupled space learning of image style transformation. In: ICCV, IEEE, pp 1, 2, 3, 4

  26. Chang KW, Hsieh CJ, Lin CJ (2008) Coordinate descent method for large-scale L2-loss linear support vector machines. J Mach Learn Res 9(7):1369–1398

    MathSciNet  MATH  Google Scholar 

  27. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32(2):407–499

    Article  MathSciNet  MATH  Google Scholar 

  28. Oreifej O, Liu Z (2013) HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. In: CVPR, Portland, Oregon, June 2013

  29. Yang X, Tian Y (2012) EigenJoints-based action recognition using naïve-bayes-nearest-neighbor. In: IEEE workshop on CVPR, pp 14–19

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (61202168, 61201234, 61100124, 21106095), the grant of Elite Scholar Program of Tianjin University, the grant of Introducing Talents to Tianjin Normal University (5RL123), the grant of Introduction of One Thousand High-level Talents in 3 years in Tianjin, Tianjin Research Program of Application Foundation and Advanced Technology (14JCZDJC31700, 13JCQNJC0040 and 20120802).

Author information

Authors and Affiliations

  1. Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology, Tianjin, 300384, People’s Republic of China

    Zan Gao, Hua Zhang, Guangping Xu & Yanbing Xue

  2. Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384, People’s Republic of China

    Zan Gao, Hua Zhang, Guangping Xu & Yanbing Xue

  3. School of Electronic Information Engineering, Tianjin University, Tianjin, 300072, People’s Republic of China

    Anan A. Liu

Authors
  1. Zan Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hua Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Anan A. Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Guangping Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yanbing Xue
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Anan A. Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, Z., Zhang, H., Liu, A.A. et al. Human action recognition on depth dataset. Neural Comput & Applic 27, 2047–2054 (2016). https://doi.org/10.1007/s00521-015-2002-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2002-0

Keywords