Skip to main content
SpringerLink
Log in

Action recognition using multi-directional projected depth motion maps

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Camera-based action recognition plays a key role in diverse computer vision applications such as human computer interaction. This paper proposes a new action recognition approach using multi-directional projected depth motion map based motion descriptors. First, for the input depth video sequence, all the depth frames in the video are projected onto multiple planes to form the projected images. The absolute difference between two consecutive projected images is accumulated through the entire depth video for establishing maps from multiple views. Then, the local motion consistency of the map is examined to form a histogram of local binary patterns, which are then concatenated and further incorporated into a kernel-based extreme learning machine for action recognition. In contrast to that only three directions are used to calculated the projected depth images for motion feature extraction in the conventional approaches, the proposed approach is able to provide an effective and flexible framework to examine the depth motion maps in multiple projected directions. The proposed approach is evaluated in the well-known MSRA action and gesture video benchmark datasets to demonstrate its superior performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Cai L, Zhu J, Zeng H, Chen J, Cai C, Ma KK (2018) HOG-assisted deep feature learning for pedestrian gender recognition. J Franklin Inst 355(4):1991–2008

    Article  Google Scholar 

  • Chaquet JM, Carmona EJ, Fernández-Caballero A (2013) A survey of video datasets for human action and activity recognition. Comput Vis Image Underst 117(6):633–659

    Article  Google Scholar 

  • Chen C, Liu K, Kehtarnavaz N (2016) Real-time human action recognition based on depth motion maps. J Real-Time Image Process 12(1):155–163

    Article  Google Scholar 

  • Chen J, Wang B, Zeng H, Cai C, Ma KK (2017) Sum-of-gradient based fast intra coding in 3d-hevc for depth map sequence. J Vis Commun Image Represent 48:329–339

    Article  Google Scholar 

  • Chen T, Chiu MC (2018) Smart technologies for assisting the life quality of persons in a mobile environment: a review. J Amb Intel Hum Comp 9(2):319–327

    Article  Google Scholar 

  • Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  • Jia C, Fu Y (2016) Low-rank tensor subspace learning for rgb-d action recognition. IEEE Trans Image Process 25(10):4641–4652

    Article  MathSciNet  ADS  Google Scholar 

  • Khaire P, Kumar P, Imran J (2018) Combining CNN streams of RGB-D and skeletal data for human activity recognition. Pat Recogn Lett. https://doi.org/10.1016/j.patrec.2018.04.035

  • Kläser A, Marszałek M, Schmid C (2008) A spatio-temporal descriptor based on 3D-gradients. In: Everingham M, Needham C, Fraile R (eds) British Machine Vision Conference, Leeds, Royaume-Uni, pp 1–10

  • Kurakin A, Zhang Z, Liu Z (2012) A real time system for dynamic hand gesture recognition with a depth sensor. In: European Signal Processing Conference, pp 1975–1979

  • Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. IEEE Conf Comput Vis Pattern Recognition:9–14

    Google Scholar 

  • Liang B, Zheng L (2015) A survey on human action recognition using depth sensors. In: Int. Conf. on Digital Image Computing: Techniques and Applications, pp 1–8

  • Liu M, Liu H, Chen C (2017) Robust 3D action recognition through sampling local appearances and global distributions. IEEE Trans Multimed. https://doi.org/10.1109/TMM.2017.2786868

  • Liu Z, Huang J, Han J, Bu S, Lv J (2017b) Human motion tracking by multiple rgbd cameras. IEEE Trans Circ Syst Video Technol 27(9):2014–2027

    Article  Google Scholar 

  • Madany N, He Y, Guan L (2016) Human action recognition via multiview discriminative analysis of canonical correlations. In: IEEE Int. Conf. on Image Processing, pp 4170–4174

  • Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  • Oreifej O, Liu Z (2013) HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences. IEEE Conf Comput Vis Pattern Recognition:716–723

    Google Scholar 

  • Sathyanarayana S, Satzoda RK, Sathyanarayana S, Thambipillai S (2018) Vision-based patient monitoring: a comprehensive review of algorithms and technologies. J Amb Intell Hum Comp 9(2):225–251

    Article  Google Scholar 

  • Tian J, Chen L (2017) Abnormal motion detection in video using statistics of spatiotemporal local kinematics pattern. In: IEEE Int. Conf. on Image Processing, Beijing, China, pp 2065–2068

  • Veeriah V, Zhuang N, Qi G (2015) Differential recurrent neural networks for action recognition. In: IEEE Int. Conf. on Computer Vision, pp 4041–4049

  • Wang J, Liu Z, Chorowski J, Chen Z, Wu Y (2012) Robust 3D action recognition with random occupancy patterns. In: European Conf. on Computer Vision, pp 872–885

  • Wang J, Liu Z, Wu Y, Yuan J (2012) Mining actionlet ensemble for action recognition with depth cameras. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp 1290–1297

  • Wang P, Li W, Gao Z, Zhang J, Tang C, Ogunbona PO (2016) Action recognition from depth maps using deep convolutional neural networks. IEEE Trans Hum Mach Syst 46(4):498–509

    Article  Google Scholar 

  • Xia L, Aggarwal JK (2013) Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp 2834–2841

  • Yang X, Zhang C, Tian Y (2012) Recognizing actions using depth motion maps-based histograms of oriented gradients. In: ACM Int. Conf. on Multimedia, pp 1057–1060

  • Yu M, Liu L, Shao L (2016) Structure-preserving binary representations for RGB-D action recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1651–1664

    Article  PubMed  Google Scholar 

  • Zhang S, Wei Z, Nie J, Huang L, Wang S, Li Z (2017) A review on human activity recognition using vision-based method. J Healthcare Eng https://doi.org/10.1155/2017/3090343

Download references

Author information

Authors and Affiliations

  1. Institute of Systems Science, National University of Singapore, Singapore, 119615, Singapore

    Sowndarya Satyamurthi, Jing Tian & Matthew Chin Heng Chua

Authors
  1. Sowndarya Satyamurthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Chin Heng Chua
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Tian.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Satyamurthi, S., Tian, J. & Chua, M.C.H. Action recognition using multi-directional projected depth motion maps. J Ambient Intell Human Comput 14, 14767–14773 (2023). https://doi.org/10.1007/s12652-018-1136-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-018-1136-1

Keywords

Search

Navigation