Simran Kumar
ABSTRACT
Human Pose Estimation, or HPE, pertains to the process of precisely determining the spatial coordinates of distinct anatomical segments of the human body depicted within an image frame. This challenge has garnered significant attention over the last decade and is poised to remain a focal point in the future. Beyond its significance within the scientific community, this field holds practical implications for applications such as augmented reality, military operations, and surveillance of densely populated areas. The present study concentrates on the intricacies of 3D HPE using a single 2D camera. The exploration encompasses the accessibility of data, prevailing metrics pertinent to this endeavour, and the complications introduced by occlusion. A specific focus is placed on introducing a methodology to evaluate the accuracy of predictions for visible and occluded key points in scenes characterized by corresponding visibility conditions. Promising outcomes are obtained, as the accuracy of predictions for visible key points remains consistent even in the presence of occlusion affecting other key points. Furthermore, a prospective avenue of research is proposed, involving the identification of occluded key points through the analysis of high-frequency noise incorporated into joint positions over temporal intervals. To conclude, an analysis is provided concerning both the challenges and opportunities inherent in this research endeavour.
- Zheng C, Wu W, Chen C, Yang T, Zhu S, Shen J, Kehtarnavaz N, Shah M (2022) Deeplearning-based human pose estimation: A survey. In: arXiv.org. (link).Google Scholar
- Li W, Liu H, Tang H, Wang P, Van Gool L. Mhformer: Multi-hypothesis transformer for 3dhuman pose estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 (pp. 13147-13156).Google Scholar
- Wang J, Yan S, Xiong Y, Lin D. Motion guided 3d pose estimation from videos. InEuropeanConference on Computer Vision 2020 Aug 23 (pp. 764-780). Springer, Cham.Google Scholar
- He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. InProceedings ofthe IEEE conference on computer vision and pattern recognition 2016 (pp. 770-778).Google Scholar
- Cai Z, Vasconcelos N. Cascade R-CNN: high quality object detection and instance segmentation. IEEE transactions on pattern analysis and machine intelligence. 2019 Nov 28;43(5):1483-98.Google Scholar
- Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).Google Scholar
- Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q. Centernet: Keypoint triplets for object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 6569-6578).Google Scholar
- Chen K, Pang J, Wang J, Xiong Y, Li X, Sun S, Feng W, Liu Z, Shi J, Ouyang W, Loy CC.Hybrid task cascade for instance segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2019 (pp. 4974-4983).Google Scholar
- Zhang J, Cai Y, Yan S, Feng J. Direct multi-view multi-person 3d pose estimation. Advancesin Neural Information Processing Systems. 2021 Dec 6;34:13153-64.Google Scholar
- Kevin Lin, Lijuan Wang, and Zicheng Liu. 2021. End-to-end human pose and mesh reconstruction with transformers. In CVPR. 1954–1963.Google Scholar
- Lin K, Wang L, Liu Z. End-to-end human pose and mesh reconstruction with transformers.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 (pp. 1954-1963).Google Scholar
- Li J, Wang C, Liu W, Qian C, Lu CH. Hierarchical multi-person ordinal relations for monocular multi-person 3d pose estimation. arXiv 2020. arXiv preprint arXiv:2008.00206.Google Scholar
- Cheng Y, Wang B, Yang B, Tan RT. Graph and temporal convolutional networks for 3dmulti-person pose estimation in monocular videos. InProceedings of the AAAI Conference on Artificial Intelligence 2021 May 18 (Vol. 35, No. 2, pp. 1157-1165).Google Scholar
- Zanfir A, Marinoiu E, Sminchisescu C. Monocular 3d pose and shape estimation of multiplepeople in natural scenes-the importance of multiple scene constraints. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018 (pp. 2148-2157).Google Scholar
- Fabbri M, Lanzi F, Calderara S, Alletto S, Cucchiara R. Compressed volumetric heatmaps formulti-person 3d pose estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 (pp. 7204-7213).Google Scholar
- Benzine A, Chabot F, Luvison B, Pham QC, Achard C. Pandanet: Anchor-based single-shotmulti-person 3d pose estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 (pp. 6856-6865).Google Scholar
- Zhen J, Fang Q, Sun J, Liu W, Jiang W, Bao H, Zhou X. Smap: Single-shot multi-personabsolute 3d pose estimation. InEuropean Conference on Computer Vision 2020 Aug 23 (pp. 550-566). Springer, Cham.Google Scholar
- Ma H, Chen L, Kong D, Wang Z, Liu X, Tang H, Yan X, Xie Y, Lin SY, Xie X. Transfusion: Cross-view fusion with transformer for 3d human pose estimation. arXiv preprint arXiv:2110.09554. 2021 Oct 18.Google Scholar
- Xie R, Wang C, Wang Y. Metafuse: A pre-trained fusion model for human pose estimation.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 (pp. 13686-13695).Google Scholar
- Ionescu C, Papava D, Olaru V, Sminchisescu C. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence. 2013 Dec 12;36(7):1325-39.Google Scholar
- Mehta D, Rhodin H, Casas D, Fua P, Sotnychenko O, Xu W, Theobalt C. Monocular 3dhuman pose estimation in the wild using improved cnn supervision. In2017 international conference on 3D vision (3DV) 2017 Oct 10 (pp. 506-516). IEEE.Google Scholar
- Mehta D, Sotnychenko O, Mueller F, Xu W, Sridhar S, Pons-Moll G, Theobalt C. Single-shotmulti-person 3d pose estimation from monocular rgb. In2018 International Conference on 3D Vision (3DV) 2018 Sep 5 (pp. 120-130). IEEE.Google Scholar
- MMPose Contributors. OpenMMLab Pose Estimation Toolbox and Benchmark, 2020. Available at: https://github.com/open-mmlab/mmpose.Google Scholar
- Girshick, R., 2015. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440-1448).Google ScholarDigital Library
- He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition.In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770778).Google Scholar
Index Terms
- Single-Camera 3D Human Pose Estimation: Addressing Occlusion Challenges and Predictive Quality Assessment
Recommendations
Multiple people tracking and pose estimation with occlusion estimation
Simultaneously tracking poses of multiple people is a difficult problem because of inter-person occlusions and self occlusions. This paper presents an approach that circumvents this problem by performing tracking based on observations from multiple wide-...
Adaptive occlusion state estimation for human pose tracking under self-occlusions
Tracking human poses in video can be considered as the process of inferring the positions of the body joints. Among various obstacles to this task, one of the most challenging is to deal with 'self-occlusion', where one body part occludes another one. ...
Real-time camera pose estimation via line tracking
Real-time camera calibration has been intensively studied in augmented reality. However, for texture-less and texture-repeated scenes as well as poorly illuminated scenes, obtaining a stable calibration is still an open problem. In the paper, we propose ...
Comments