skip to main content
10.1145/3197768.3197791acmotherconferencesArticle/Chapter ViewAbstractPublication PagespetraConference Proceedingsconference-collections
research-article

A Comparative Study of Matrix Completion and Recovery Techniques for Human Pose Estimation

Published: 26 June 2018 Publication History

Abstract

We present a comparative study of three matrix completion and recovery techniques, applied to the problem of human pose estimation. Human pose estimation algorithms may exhibit estimation noise or may completely fail to provide estimates for some joints. A post-process is often employed to recover the missing joints' locations from the available ones, typically by enforcing kinematic constraints or by using a prior learned from a database of natural poses. Matrix completion and recovery techniques fall into the latter category and operate by filling-in missing entries of a matrix, with the available/non-missing entries being potentially corrupted by noise. We compare the performance of three such techniques in terms of the estimation error of their output as well as their runtime under varying parameters. We conclude by recommending use cases for each of the compared techniques.

References

[1]
Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2014. 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. In Computer Vision and Pattern Recognition. 3686--3693.
[2]
Andreas Baak, Meinard Muller, Gaurav Bharaj, Hans Peter Seidel, and Christian Theobalt. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. International Conference on Computer Vision (2011), 1092--1099.
[3]
Ernesto Brau and Hao Jiang. 2016. 3D Human Pose Estimation via Deep Learning from 2D Annotations. International Conference on 3D Vision (2016).
[4]
Emmanuel J. Candès and Benjamin Recht. 2009. Exact matrix completion via convex optimization. Foundations of Computational Mathematics 9, 6 (2009), 717--772. arXiv:0805.4471
[5]
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. (2017). arXiv:1611.08050
[6]
Simone Ciotti, Edoardo Battaglia, Iason Oikonomidis, Alexandros Makris, Aggeliki Tsoli, Antonio Bicchi, Antonis A Argyros, and Matteo Bianchi. 2016. Synergy-driven Performance Enhancement of Vision-based 3D Hand Pose Reconstruction. In International Conference on Wireless Mobile Communication and Healthcare (MobiHealth 2016), special session on advances in soft wearable technology for mobile-health. 1--8.
[7]
Petros Douvantzis, Iason Oikonomidis, Nikolaos Kyriazis, and Antonis A Argyros. 2013. Dimensionality Reduction for Efficient Single Frame Hand Pose Estimation. In International Conference on Computer Vision Systems (ICVS 2013). Springer, St. Petersburg, Russia, 143--152.
[8]
A. Elhayek, E. De Aguiar, A. Jain, J. Thompson, L. Pishchulin, M. Andriluka, C. Bregler, B. Schiele, and C. Theobalt. 2017. MARCOnI - ConvNet-Based MARkerless motion capture in outdoor and indoor scenes. Pattern Analysis and Machine Intelligence, IEEE Transactions on 39, 3 (2017), 501--514.
[9]
Ali Erol, George Bebis, Mircea Nicolescu, Richard D. Boyle, and Xander Twombly. 2007. Vision-based hand pose estimation: A review. Computer Vision and Image Understanding 108, 1-2 (2007), 52--73.
[10]
Shachar Fleishman, Mark Kliger, Alon Lerner, and Gershom Kutliroff. 2015. ICPIK: Inverse Kinematics based articulated-ICP. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Vol. 2015-Octob. 28--35.
[11]
Michael Foukarakis, Ilia Adami, Danae Ioannidi, Asterios Leonidis, Damien Michel, Ammar Qammaz, Konstantinos Papoutsakis, Margherita Antona, and Antonis A Argyros. 2016. A Robot-based Application for Physical Exercise Training. In International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2016). Scitepress, Rome, Italy, 45--52.
[12]
Berthold K. P. Horn. 1987. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 4 (1987), 629.
[13]
Eldar Insafutdinov, Leonid Pishchulin, Bjoern Andres, Mykhaylo Andriluka, and Bernt Schiele. 2016. Deepercut: A deeper, stronger, and faster multi-person pose estimation model. In European Conference on Computer Vision. arXiv:1605.03170
[14]
Sam Johnson and Mark Everingham. 2011. Learning effective human pose estimation from inaccurate annotation. Computer Vision and Pattern Recognition (2011), 1465--1472.
[15]
Hanbyul Joo, Tomas Simon, and Yaser Sheikh. 2018. Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies. arXiv preprint arXiv:1801.01615 (2018). arXiv:1801.01615
[16]
Nikolaos Kyriazis and Antonis A Argyros. 2013. Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis. In IEEE Computer Vision and Pattern Recognition (CVPR 2013). IEEE, Portland, Oregon, USA, 9--16.
[17]
Yann Lecun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436--444. arXiv:arXiv:1312.6184v5
[18]
Ita Lifshitz, Ethan Fetaya, and Shimon Ullman. 2016. Human Pose Estimation using Deep Consensus Voting. In European Conference on Computer Vision. arXiv:1603.08212
[19]
Zhouchen Lin, Minming Chen, and Yi Ma. 2010. The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices. (2010). arXiv:1009.5055
[20]
Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. In ACM Transactions on Graphics (SIGGRAPH 2017). arXiv:1705.01583
[21]
Stan Melax, Leonid Keselman, and Sterling Orsten. 2013. Dynamics Based 3D Skeletal Hand Tracking. In Proc. of Graphics Interface. arXiv:1705.07640 http://arxiv.org/abs/1705.07640
[22]
Damien Michel and Antonis A Argyros. 2016. Apparatuses, methods and systems for recovering a 3-dimensional skeletal model of the human body. (24 March 2016).
[23]
Damien Michel, Ammar Qammaz, and Antonis A Argyros. 2017. Markerless 3D Human Pose Estimation and Tracking based on RGBD Cameras: an Experimental Evaluation. In International Conference on Pervasive Technologies Related to Assistive Environments (PETRA 2017). ACM, Rhodes, Greece, 115--122.
[24]
Thomas B. Moeslund and Erik Granum. 2001. A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 81, 3 (2001), 231--268.
[25]
Francesc Moreno-Noguer. 2016. 3D Human Pose Estimation from a Single Image via Distance Matrix Regression. In Computer Vision and Pattern Recognition. arXiv:1611.09010
[26]
Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked Hourglass Networks for Human Pose Estimation. In European Conference on Computer Vision. arXiv:1603.06937
[27]
Markus Oberweger and Vincent Lepetit. 2017. DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation. In ICCV workshop, Vol. 840. arXiv:1708.08325
[28]
Markus Oberweger, Paul Wohlhart, and Vincent Lepetit. 2015. Hands Deep in Deep Learning for Hand Pose Estimation. In Computer Vision Winter Workshop. arXiv:1502.06807
[29]
Ferda Ofli, Rizwan Chaudhry, Gregorij Kurillo, Rene Vidal, and Ruzena Bajcsy. 2013. Berkeley MHAD: A comprehensive Multimodal Human Action Database. Proceedings of IEEE Workshop on Applications of Computer Vision (2013), 53--60.
[30]
OptiTrack. {n. d.}. OptiTrack - Motion Capture Systems. ({n. d.}). http://optitrack.com/
[31]
Art B. Owen and Patrick O. Perry. 2009. Bi-cross-validation of the SVD and the nonnegative matrix factorization. Annals of Applied Statistics 3, 2 (2009), 564--594. arXiv:0908.2062
[32]
Paschalis Panteleris and Antonis A Argyros. 2016. Monitoring and Interpreting Human Motion to Support Clinical Applications of a Smart Walker. In Workshop on Human Motion Analysis for Healthcare Applications (HMAHA 2016). IET, London, UK.
[33]
Georgios Pavlakos, Xiaowei Zhou, Konstantinos G. Derpanis, and Kostas Daniilidis. 2016. Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose. In Computer Vision and Pattern Recognition. arXiv:1611.07828
[34]
Microsoft Corp. Redmond. {n. d.}. Kinect for Xbox 360. ({n. d.}).
[35]
Konstantinos Roditakis, Alexandros Makris, and Antonis A Argyros. 2017. Generative 3D Hand Tracking with Spatially Constrained Pose Sampling. In British Machine Vision Conference (BMVC 2017). BMVA, London, UK.
[36]
Javier Romero, Dimitrios Tzionas, and Michael J. Black. 2017. Embodied hands: Modeling and Capturing Hands and Bodies Together. ACM Transactions on Graphics (SIGGRAPH Asia 2017) 36, 6 (2017), 1--17.
[37]
Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby Sharp, Mark Finocchio, Richard Moore, Alex Kipman, and Aandrew Blake. 2011. Real-Time Human Pose Recognition in Parts from Single Depth Images. Computer Vision and Pattern Recognition (2011). http://research.microsoft.com/pubs/145347/BodyPartRecognition.pdf
[38]
E Simo-Serra, C Torras, and F Moreno-Noguer. 2015. Lie algebra-based kinematic prior for 3D human pose tracking. Machine Vision Applications (MVA), International Conference on (2015), 0--3.
[39]
Ayan Sinha, Chiho Choi, and Karthik Ramani. 2016. DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed with Deep Features. In Computer Vision and Pattern Recognition. 4150--4158. arXiv:arXiv:1011.1669v3
[40]
Xiao Sun, Yichen Wei, Shuang Liang, Xiaoou Tang, and Jian Sun. 2015. Cascaded hand pose regression. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 07-12-June. 824--832.
[41]
D. Tang, J. Taylor, P. Kohli, C. Keskin, T. K. Kim, and J. Shotton. 2015. Opening the Black Box: Hierarchical Sampling Optimization for Estimating Human Hand Pose. In ICCV. 3325--3333.
[42]
Bugra Tekin, Isinsu Katircioglu, Mathieu Salzmann, Vincent Lepetit, and Pascal Fua. 2016. Structured Prediction of 3D Human Pose with Deep Neural Networks. In British Machine Vision Conference. arXiv:1605.05180 http://arxiv.org/abs/1605.05180
[43]
Bugra Tekin, Artem Rozantsev, Vincent Lepetit, and Pascal Fua. 2016. Direct Prediction of 3D Body Poses from Motion Compensated Sequences. In Computer Vision and Pattern Recognition. arXiv:1511.06692
[44]
Jonathan Tompson, Murphy Stein, Yann Lecun, and Ken Perlin. 2014. Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks. ACM Transactions on Graphics (SIGGRAPH 2014) 33, 5 (2014), 1--10.
[45]
Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Marc Pollefeys, and Juergen Gall. 2016. Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation. International Journal of Computer Vision 118, 2 (2016), 172--193. arXiv:1506.02178
[46]
Vicon. {n. d.}. Motion Capture Systems | Vicon. ({n. d.}). https://www.vicon.com/
[47]
Robert Y. Wang and Jovan Popović. 2009. Real-time hand-tracking with a color glove. ACM Transactions on Graphics 28, 3 (jul 2009), 1.
[48]
Tao Yu, Kaiwen Guo, Feng Xu, Yuan Dong, Zhaoqi Su, Jianhui Zhao, Jianguo Li, Qionghai Dai, and Yebin Liu. 2017. BodyFusion: Real-time Capture of Human Motion and Surface Geometry Using a Single Depth Camera. In International Conference on Computer Vision.
[49]
Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, and Yichen Wei. 2017. Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach. In International Conference on Computer Vision. arXiv:1704.02447

Cited By

View all
  • (2024)A Fast and Efficient Approach for Human Action Recovery From Corrupted 3-D Motion Capture Data Using QR Decomposition-Based Approximate SVDIEEE Transactions on Human-Machine Systems10.1109/THMS.2024.340029054:4(395-405)Online publication date: Aug-2024
  • (2022)A fast non-convex optimization technique for human action recovery from misrepresented 3D motion capture data using trajectory movement and pair-wise hierarchical constraintsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-022-04349-z14:8(10779-10797)Online publication date: 14-Aug-2022
  • (2020) l 1/2 Regularized RPCA Technique for 3D Human Action Recovery 2020 IEEE 17th India Council International Conference (INDICON)10.1109/INDICON49873.2020.9342124(1-5)Online publication date: 10-Dec-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference
June 2018
591 pages
ISBN:9781450363907
DOI:10.1145/3197768
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

In-Cooperation

  • NSF: National Science Foundation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Matrix completion
  2. comparative study
  3. human pose estimation
  4. matrix recovery

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

PETRA '18

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)2
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Fast and Efficient Approach for Human Action Recovery From Corrupted 3-D Motion Capture Data Using QR Decomposition-Based Approximate SVDIEEE Transactions on Human-Machine Systems10.1109/THMS.2024.340029054:4(395-405)Online publication date: Aug-2024
  • (2022)A fast non-convex optimization technique for human action recovery from misrepresented 3D motion capture data using trajectory movement and pair-wise hierarchical constraintsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-022-04349-z14:8(10779-10797)Online publication date: 14-Aug-2022
  • (2020) l 1/2 Regularized RPCA Technique for 3D Human Action Recovery 2020 IEEE 17th India Council International Conference (INDICON)10.1109/INDICON49873.2020.9342124(1-5)Online publication date: 10-Dec-2020
  • (2018)Filling the Joints: Completion and Recovery of Incomplete 3D Human PosesTechnologies10.3390/technologies60400976:4(97)Online publication date: 30-Oct-2018

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media