skip to main content
10.1145/2915926.2915937acmotherconferencesArticle/Chapter ViewAbstractPublication PagescasaConference Proceedingsconference-collections
short-paper

Joint-Triplet Motion Image and Local Binary Pattern for 3D Action Recognition Using Kinect

Published: 23 May 2016 Publication History

Abstract

This paper presents a new action recognition method that utilizes the 3D skeletal motion data captured using the Kinect depth sensor. We propose a robust view-invariant joint motion representation based on the spatio-temporal changes in relative angles among the different skeletal joint-triplets, namely the joint relative angle (JRA). A sequence of JRAs obtained for a particular joint-triplet intuitively represents the level of involvement of those joints in performing a specific action. Collection of all joint-triplet JRA sequences is then utilized to construct a spatial holistic description of action-specific motion patterns, namely the 2D joint-triplet motion image. The proposed method exploits a local texture analysis method, the local binary pattern (LBP), to highlight micro-level texture details in the motion images. This process isolates prototypical features for different actions. LBP histogram features are then projected into a discriminant Fisher-space, resulting in more compact and disjoint feature clusters representing individual actions. The performance of the proposed method is evaluated using two publicly available Kinect action databases. Extensive experiments show advantage of the proposed joint-triplet motion image and LBP-based action recognition approach over existing methods.

References

[1]
F. Ahmed. Gradient directional pattern: a robust feature descriptor for facial expression recognition. IET Electronics Letters, 48(19):1203--1204, 2012.
[2]
F. Ahmed, P. Paul, and M. Gavrilova. Dtw-based kernel and rank level fusion for 3d gait recognition using kinect. The Visual Computer, 31(6-8):915--924, 2015.
[3]
F. Ahmed, P. Paul, and M. Gavrilova. Evolutionary fusion of local texture patterns for facial expression recognition. In IEEE International Conference on Image Processing (ICIP), pages 1031--1035, 2015.
[4]
F. Ahmed, P. Paul, M. Gavrilova, and R. Alhajj. Weighted fusion of bit plane-specific local image descriptors for facial expression recognition. In IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 1852--1857, 2015.
[5]
M. Bae and I. Park. Content-based 3d model retrieval using a single depth image from a low-cost 3d camera. The Visual Computer, 29:555--564, 2013.
[6]
A. Ball, D. Rye, F. Ramos, and M. Velonaki. Unsupervised clustering of people from 'skeleton' data. In ACM/IEEE International Conference on Human Robot Interaction, pages 225--226, 2012.
[7]
M. Barnachona, S. Bouakaza, B. Boufamab, and E. Guilloua. Ongoing human action recognition with motion capture. Pattern Recognition, 47(1):238--247, 2014.
[8]
P. N. Belhumeur, J. P. Hespanha, and D. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711--720, 1997.
[9]
A. Bobick and J. Davis. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machince Intelligence, 23(3):257--267, 2001.
[10]
J. Botiaa, A. Villab, and J. Palmaa. Ambient assisted living system for in-home monitoring of healthy independent elders. Expert Systems with Applications, 39(9):8136--8148, 2012.
[11]
Y. Chang, S. Chen, and J. Huang. A kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 32(6):2566--2570, 2011.
[12]
C. de Boor. A Practical Guide to Splines. New York: Springer Verlag, 1978.
[13]
A. Efros, A. Berg, G. Mori, and J. Malik. Recognizing action at a distance. In IEEE International Conference on Computer Vision, volume 2, pages 726--733, 2003.
[14]
M. Gabel, R. Gilad-Bachrach, E. Renshaw, and A. Schuster. Full body gait analysis with kinect. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 1964--1967, 2012.
[15]
S. Gaglio, G. L. Re, and M. Morana. Human activity recognition process using 3-d posture data. IEEE Transactions on Human-Machine Systems, 45(5):586--597, 2014.
[16]
I. Junejo, E. Dexter, I. Laptev, and P. Perez. View-independent action recognition from temporal self-similarities. IEEE Transactions on Pattern Analysis and Machince Intelligence, 33(1):172--185, 2011.
[17]
I. Laptev. On space-time interest points. International Journal of Computer Vision, 64(2/3):107--123, 2005.
[18]
I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1--8, 2008.
[19]
T. Moeslund, A. Hilton, and V. Kruger. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding, 104(2-3):90--126, 2006.
[20]
P. Natarajan and R. Nevatia. View and scale invariant action recognition using multiview shape-flow models. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1--8, 2008.
[21]
T. Ojala, M. Pietikainen, and T. Maenpaa. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machince Intelligence, 24(7):971--987, 2002.
[22]
M. Popa, A. Koc, L. Rothkrantz, C. Shan, and P. Wiggers. Kinect sensing of shopping related actions. Communications in Computer and Information Science, (277):91--100, 2012.
[23]
R. Poppe. A survey on vision-based human action recognition. Image and Vision Computing, 28:976--990, 2010.
[24]
J. Preis, M. Kessel, C. Linnhoff-Popien, and M. Werner. Gait recognition with kinect. In Workshop on Kinect in Pervasive Computing, 2012.
[25]
E. Stone and M. Skubic. Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In International Pervasive Computing Technologies for Healthcare Conference, pages 71--77, 2011.
[26]
Y. Tian, T. Kanade, and J. Cohn. Evaluation of gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 229--234, 2002.
[27]
J. Wang, Z. Liu, Y. Wu, and J. Yuan. Learning actionlet ensemble for 3d human action recognition. IEEE Transactions on Pattern Analysis and Machince Intelligence, 36(5):914--927, 2014.
[28]
L. Xia, C. Chen, and J. Aggarwal. View invariant human action recognition using histograms of 3d joints. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 20--27, 2012.
[29]
J. Yamato, J. Ohya, and K. Ishii. Recognizing human action in timesequential images using hidden markov model. In IEEE Conference on Computer Vision and Pattern Recognition, pages 379--385, 1992.
[30]
X. Yang and Y. Tian. Effective 3d action recognition using eigenjoints. Journal of Visual Communication and Image Representation, 25(1):2--11, 2014.
[31]
Y. Zhang, J. Zheng, and N. Magnenat-Thalmann. Example-guided anthropometric human body modeling. The Visual Computer, pages 1--17, 2014.
[32]
L. Zhou, L. Zhiwu, H. Leung, and L. Shang. Spatial temporal pyramid matching using temporal sparse representation for human motion retrieval. The Visual Computer (CGI 2014), 30:845--854, 2014.

Cited By

View all
  • (2021)Residual connection-based graph convolutional neural networks for gait recognitionThe Visual Computer10.1007/s00371-021-02245-9Online publication date: 16-Jul-2021
  • (2020)Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient RehabilitationResearch Anthology on Rehabilitation Practices and Therapy10.4018/978-1-7998-3432-8.ch032(653-678)Online publication date: 21-Aug-2020
  • (2019)Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient RehabilitationDesign and Implementation of Healthcare Biometric Systems10.4018/978-1-5225-7525-2.ch007(160-184)Online publication date: 2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
CASA '16: Proceedings of the 29th International Conference on Computer Animation and Social Agents
May 2016
200 pages
ISBN:9781450347457
DOI:10.1145/2915926
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 3D skeleton
  2. action recognition
  3. joint-triplet motion image
  4. local texture descriptor
  5. spatio-temporal motion patterns

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Conference

CASA '16
CASA '16: Computer Animation and Social Agents
May 23 - 25, 2016
Geneva, Switzerland

Acceptance Rates

Overall Acceptance Rate 18 of 110 submissions, 16%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 08 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Residual connection-based graph convolutional neural networks for gait recognitionThe Visual Computer10.1007/s00371-021-02245-9Online publication date: 16-Jul-2021
  • (2020)Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient RehabilitationResearch Anthology on Rehabilitation Practices and Therapy10.4018/978-1-7998-3432-8.ch032(653-678)Online publication date: 21-Aug-2020
  • (2019)Multi-Modal Motion-Capture-Based Biometric Systems for Emergency Response and Patient RehabilitationDesign and Implementation of Healthcare Biometric Systems10.4018/978-1-5225-7525-2.ch007(160-184)Online publication date: 2019
  • (2019)Localized Trajectories for 2D and 3D Action RecognitionSensors10.3390/s1916350319:16(3503)Online publication date: 10-Aug-2019
  • (2018)Gait Type Analysis Using Dynamic Bayesian NetworksSensors10.3390/s1810332918:10(3329)Online publication date: 4-Oct-2018
  • (2018)Kinect Sensor Gesture and Activity Recognition: New Applications for Consumer Cognitive SystemsIEEE Consumer Electronics Magazine10.1109/MCE.2017.27554987:1(88-94)Online publication date: Jan-2018
  • (2018)Detection of Asymmetric Abnormalities in Gait using Depth Data and Dynamic Bayesian Networks2018 14th IEEE International Conference on Signal Processing (ICSP)10.1109/ICSP.2018.8652291(762-767)Online publication date: Aug-2018
  • (2017)Human Identification Using Gait Skeletal Joint Distance FeaturesInternational Journal of Software Science and Computational Intelligence10.4018/IJSSCI.20171001029:4(19-33)Online publication date: 1-Oct-2017
  • (2017)Utilizing gait traits to improve e-border watchlist performance2017 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2017.8285250(1-8)Online publication date: Nov-2017
  • (2017)Kinect gait skeletal joint feature-based person identification2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC)10.1109/ICCI-CC.2017.8109783(423-430)Online publication date: Jul-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media