Abstract
We present spectral graph skeletons (SGS), a novel graph-based method for action recognition from depth cameras. The contribution of this paper is to leverage a spectral graph wavelet transform (SGWT) for creating an overcomplete representation of an action signal lying on a 3D skeleton graph. The resulting SGS descriptor is efficiently computable in time linear in the action sequence length. We investigate the suitability of our method by experiments on three publicly available datasets, resulting in performance comparable to state-of-the-art action recognition approaches. Namely, our method achieves \(91.4\)% accuracy on the challenging MSRAction3D dataset in the cross-subject setting. SGS also achieves \(96.0\,\%\) and \(98.8\,\%\) accuracy on the MSRActionPairs3D and UCF-Kinect datasets, respectively. While this study focuses on action recognition, the proposed framework can in general be applied to any time series of graphs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Online source code available at http://wiki.epfl.ch/sgwt.
- 2.
In stratified cross-validation, the folds are selected so that the percentage of samples for each class in the dataset is preserved in each fold.
References
Bunke, H., Riesen, K.: Recent advances in graph-based pattern recognition with applications in document analysis. Pattern Recogn. 44, 1057–1067 (2011)
Bunke, H., Riesen, K.: Towards the unification of structural and statistical pattern recognition. Pattern Recogn. Lett. 33, 811–825 (2012)
Chen, S., Cerda, F., Rizzo, P., Bielak, J., Garrett, J., Kovacevic, J.: Semi-supervised multiresolution classification using adaptive graph filtering with application to indirect bridge structural health monitoring. IEEE Trans. Signal Proc. 62, 2879–2893 (2014)
Chung, F.R.: Spectral graph theory. CBMS Regional Conference Seriesin Mathematics, vol. 92, American Mathematical Society (1997)
Coifman, R.R., Maggioni, M.: Diffusion wavelets. Appl. Comput. Harmonic Anal. 21, 53–94 (2006)
Crovella, M., Kolaczyk, E.: Graph wavelets for spatial traffic analysis. In: INFOCOM (2003)
Shuman, D.I., Narang, S.K., Frossard, P., Ortega, A., Vandergheynst, P.: The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process. Mag. 30, 83–98 (2013)
Dong, X., Ortega, A., Frossard, P., Vandergheynst, P.: Inference of mobility patterns via spectral graph wavelets. In: ICASSP (2013)
Egilmez, H.E., Ortega, A.: Spectral anomaly detection using graph-based filtering for wireless sensor networks. In: ICASSP (2014)
Ellis, C., Masood, S.Z., Tappen, M.F., Laviola Jr., J.J., Sukthankar, R.: Exploring the trade-off between accuracy and observational latency in action recognition. Int. J. Comput. Vis. 101, 420–436 (2013)
Gowayyed, M.A., Torki, M., Hussein, M.E., El-Saban, M.: Histogram of oriented displacements (hod): describing trajectories of human joints for action recognition. In: IJCAI (2013)
Hammond, D.K., Vandergheynst, P., Gribonval, R.: Wavelets on graphs via spectral graph theory. Appl. Comput. Harmonic Anal. 30, 129–150 (2011)
Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Trans. Cybern. 43, 1318–1334 (2013)
Hermansson, L., Kerola, T., Johansson, F., Jethava, V., Dubhashi, D.: Entity disambiguation in anonymized graphs using graph kernels. In: CIKM. ACM (2013)
Kim, W.S., Narang, S.K., Ortega, A.: Graph based transforms for depth video coding. In: ICASSP (2012)
Leonardi, N., Van De Ville, D.: Wavelet frames on graphs defined by fmri functional connectivity. In: ISBI (2011)
Li, W., Zhang, Z., Liu, Z.: Action recognition based on a bag of 3d points. In: CVPR Workshops (2010)
Luo, J., Wang, W., Qi, H.: Group sparsity and geometry constrained dictionary learning for action recognition from depth maps. In: ICCV (2013)
Narang, S.K., Chao, Y.H., Ortega, A.: Graph-wavelet filterbanks for edge-aware image processing. In: Statistical Signal Processing Workshop (SSP). IEEE (2012)
Narang, S.K., Ortega, A.: Perfect reconstruction two-channel wavelet filter banks for graph structured data. IEEE Trans. Signal Process. 60, 2786–2799 (2012)
Oreifej, O., Liu, Z., Redmond, W.: Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences. In: CVPR (2013)
Ram, I., Elad, M., Cohen, I.: Generalized tree-based wavelet transform. IEEE Trans. Signal Process. 59, 4199–4209 (2011)
Sandryhaila, A., Moura, J.M.F.: Nearest-neighbor image model. In: ICIP (2012)
Sandryhaila, A., Moura, J.M.F.: Discrete signal processing on graphs. IEEE Trans. Signal Process. 61, 1644–1656 (2013)
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56, 116–124 (2013)
Shuman, D.I., Vandergheynst, P., Frossard, P.: Chebyshev polynomial approximation for distributed signal processing. In: DCOSS (2011)
Thanou, D., Shuman, D.I., Frossard, P.: Parametric dictionary learning for graph signals. In: IEEE GlobalSIP (2013)
Vieira, A.W., Nascimento, E.R., Oliveira, G.L., Liu, Z., Campos, M.F.M.: STOP: Space-Time Occupancy Patterns for 3D Action Recognition from Depth Map Sequences. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds.) CIARP 2012. LNCS, vol. 7441, pp. 252–259. Springer, Heidelberg (2012)
Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: CVPR (2012)
Wang, J., Wu, Y.: Learning maximum margin temporal warping for action recognition. In: ICCV (2013)
Xia, L., Chen, C.C., Aggarwal, J.: View invariant human action recognition using histograms of 3d joints. In: CVPR Workshops (2012)
Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR (2009)
Yang, X., Tian, Y.: Eigenjoints-based action recognition using naive-bayes-nearest-neighbor. In: CVPR Workshops (2012)
Yang, X., Zhang, C., Tian, Y.: Recognizing actions using depth motion maps-based histograms of oriented gradients. In: ACM MM (2012)
Zanfir, M., Leordeanu, M., Sminchisescu, C.: The moving pose: An efficient 3d kinematics descriptor for low-latency action recognition and detection. In: ICCV (2013)
Zhao, X., Li, X., Pang, C., Zhu, X., Sheng, Q.Z.: Online human gesture recognition from motion data streams. In: ACM MM (2013)
Zhu, X., Rabbat, M.: Approximating signals supported on graphs. In: ICASSP (2012)
Zhu, X., Kandola, J., Lafferty, J., Ghahramani, Z.: Graph kernels by spectral transforms. In: Chapelle, O., Schoelkopf, B., Zien, A. (eds.) Semi-Supervised Learning, pp. 277–291. MIT Press (2006)
Acknowledgement
The first author acknowledges the Japanese Government (Monbukagakusho:MEXT) scholarship support for carrying out this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kerola, T., Inoue, N., Shinoda, K. (2015). Spectral Graph Skeletons for 3D Action Recognition. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9006. Springer, Cham. https://doi.org/10.1007/978-3-319-16817-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-16817-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16816-6
Online ISBN: 978-3-319-16817-3
eBook Packages: Computer ScienceComputer Science (R0)