Skip to main content
SpringerLink
Log in

Ensemble of Classifiers Using CNN and Hand-Crafted Features for Depth-Based Action Recognition

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11509))

Included in the following conference series:

Abstract

In this paper, we present an algorithm for action recognition that uses only depth maps. At the beginning we extract features describing the person shape in single depth maps. For each class we train a separate one-against-all convolutional neural network to extract class-specific features. The actions are represented by multivariate time-series of such CNN-based frame features for which we calculate statistical features. For the non-zero pixels representing the person shape in each depth map we calculate handcrafted features. For time-series of such handcrafted features we calculate the statistical features. Afterwards, handcrafted features that are common for all actions and CNN-based features that are action-specific are concatenated together resulting in action feature vectors. For each action feature vector we train a multi-class classifier with one-hot encoding of output labels. The prediction of the action is done by a voting-based ensemble operating on such one-hot encoding outputs. We demonstrate experimentally that on UTD-MHAD dataset the proposed algorithm outperforms state-of-the-art depth-based algorithms and achieves promising results on MSR-Action3D dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/tjacek/DeepActionLearning.

References

  1. Aggarwal, J., Ryoo, M.: Human activity analysis: a review. ACM Comput. Surv. 43(3), 16:1–16:43 (2011)

    Article  Google Scholar 

  2. Liang, B., Zheng, L.: A survey on human action recognition using depth sensors. In: International Conference on Digital Image Computing: Techniques and Applications, pp. 1–8 (2015)

    Google Scholar 

  3. Aggarwal, J., Xia, L.: Human activity recognition from 3D data: a review. Pattern Recognit. Lett. 48, 70–80 (2014)

    Article  Google Scholar 

  4. Chen, L., Wei, H., Ferryman, J.: A survey of human motion analysis using depth imagery. Pattern Recognit. Lett. 34(15), 1995–2006 (2013)

    Article  Google Scholar 

  5. Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., Gall, J.: A survey on human motion analysis from depth data. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications. LNCS, vol. 8200, pp. 149–187. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-44964-2_8

    Chapter  Google Scholar 

  6. Lo Presti, L., La Cascia, M.: 3D skeleton-based human action classification. Pattern Recognit. 53(C), 130–147 (2016)

    Article  Google Scholar 

  7. Xia, L., Chen, C.C., Aggarwal, J.: View invariant human action recognition using histograms of 3D joints. In: CVPR Workshops, pp. 20–27 (2012)

    Google Scholar 

  8. Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3D points. In: IEEE International Conference on Computer Vision and Pattern Recognition - Workshops, pp. 9–14 (2010)

    Google Scholar 

  9. Chen, C., Jafari, R., Kehtarnavaz, N.: Action recognition from depth sequences using depth motion maps-based local binary patterns. In: 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 1092–1099 (2015)

    Google Scholar 

  10. Yang, X., Zhang, C., Tian, Y.L.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 1057–1060. ACM (2012)

    Google Scholar 

  11. Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y.: Robust 3D action recognition with random occupancy patterns. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 872–885. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_62

    Chapter  Google Scholar 

  12. Vieira, A.W., Nascimento, E.R., Oliveira, G.L., Liu, Z., Campos, M.F.M.: STOP: space-time occupancy patterns for 3D action recognition from depth map sequences. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds.) CIARP 2012. LNCS, vol. 7441, pp. 252–259. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33275-3_31

    Chapter  Google Scholar 

  13. Xia, L., Aggarwal, J.: Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2834–2841 (2013)

    Google Scholar 

  14. Oreifej, O., Liu, Z.: HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 716–723 (2013)

    Google Scholar 

  15. Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., Ogunbona, P.: Action recognition from depth maps using deep convolutional neural networks. IEEE Trans. Hum.-Mach. Syst. 46(4), 498–509 (2016)

    Article  Google Scholar 

  16. Chen, C., Jafari, R., Kehtarnavaz, N.: UTD-MHAD: a multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 168–172, September 2015

    Google Scholar 

  17. Wang, P., Li, W., Li, C., Hou, Y.: Action recognition based on joint trajectory maps with convolutional neural networks. Knowl.-Based Syst. 158, 43–53 (2018)

    Article  Google Scholar 

  18. Hou, Y., Li, Z., Wang, P., Li, W.: Skeleton optical spectra-based action recognition using convolutional neural networks. IEEE Trans. Circuits Syst. Video Technol. 28(3), 807–811 (2018)

    Article  Google Scholar 

  19. Hussein, M.E., Torki, M., Gowayyed, M.A., El-Saban, M.: Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations. In: Proceedings of the Twenty-Third International Joint Conferences on Artificial Intelligence, IJCAI 2013, pp. 2466–2472. AAAI Press (2013)

    Google Scholar 

  20. Zhou, L., Li, W., Zhang, Y., Ogunbona, P., Nguyen, D., Zhang, H.: Discriminative key pose extraction using extended LC-KSVD for action recognition. In: 2014 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2014)

    Google Scholar 

  21. Wang, P., Wang, S., Gao, Z., Hou, Y., Li, W.: Structured images for RGB-D action recognition. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 1005–1014 (2017)

    Google Scholar 

  22. Wu, Y.: Mining actionlet ensemble for action recognition with depth cameras. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1290–1297 (2012)

    Google Scholar 

  23. Wang, P., Li, W., Gao, Z., Zhang, J., Tang, C., Ogunbona, P.O.: Action recognition from depth maps using deep convolutional neural networks. IEEE Trans. Hum.-Mach. Syst. 46(4), 498–509 (2016)

    Article  Google Scholar 

  24. Lu, C., Jia, J., Tang, C.: Range-sample depth feature for action recognition. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 772–779 (2014)

    Google Scholar 

Download references

Acknowledgment

This work was supported by Polish National Science Center (NCN) under a research grant 2017/27/B/ST6/01743.

Author information

Authors and Affiliations

  1. AGH University of Science and Technology, 30 Mickiewicza, 30-059, Krakow, Poland

    Jacek Trelinski & Bogdan Kwolek

Authors
  1. Jacek Trelinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bogdan Kwolek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Kwolek .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Trelinski, J., Kwolek, B. (2019). Ensemble of Classifiers Using CNN and Hand-Crafted Features for Depth-Based Action Recognition. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2019. Lecture Notes in Computer Science(), vol 11509. Springer, Cham. https://doi.org/10.1007/978-3-030-20915-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20915-5_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20914-8

  • Online ISBN: 978-3-030-20915-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation