Abstract
This paper describes a method for prediction of 3D human pose using an RGB-D camera in real-time for personalized robot services. In input frame from RGB-D camera, 2D pose as a skeleton represented by 2D keypoints on an RGB image is estimated by OpenPose: an image-based human pose estimation technique. 3D pose represented by a set of 3D keypoints is estimated from the depth at the keypoints of 2D keypoints. Daily actions of a person, which are represented by a sequence of 3D keypoints, are measured by an RGB-D camera fixed to a daily environment. After the sequences of measured 3D keypoints are learned by a motion predictor of RNN, the future 3D keypoints are predicted by the RNN from the sequential input of 3D keypoints, which are measured in real-time by RGB-D camera. Experimental results show the predicted 3D pose and errors for standing from and siting on a chair in a room of daily environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Pishchulin, L., et al.: DeepCut: joint subset partition and labeling for multi person pose estimation. In: CVPR2016, pp. 4929–4937 (2016)
Cao, Z., Simon, T., Wei, S., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR2017, pp. 1302–1310 (2017)
Papandreou, G., Zhu, T., Chen, L., Gidaris, S., Tompson, J., Murphy, K.: PersonLab: person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11218, pp. 282–299. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_17
Chen, C.-H., Ramanan, D.: 3D human pose estimation = 2D pose estimation + matching. In: CVPR2017, pp. 5759–5767 (2017)
Pavlakos, G., Zhou, X., Derpanis, K.G., Daniilidis, K.: Coarse-to-fine volumetric prediction for single-image 3D human pose. In: CVPR2017, pp. 1263–1272 (2017)
Zhou, X., Huang, Q., Sun, X., Xue, X., Wei, Y.: Towards 3D human pose estimation in the wild: a weakly-supervised approach. In: ICCV2017, pp. 398–407 (2017)
Novotny, D., Ravi, N., Graham, B., Neverova, N., Vedaldi, A.: C3DPO: canonical 3D pose networks for non-rigid structure from motion. In: ICCV2019, pp. 7687–7696 (2019)
Cheng, Y., Yang, B., Wang, B., Yan, W., Tan, R.T.: Occlusion-aware networks for 3D human pose estimation in video. In: ICCV2019, pp. 723–732 (2019)
Moon, G., Chang, J.Y., Lee, K.M.: Camera distance-aware top-down approach for 3D multi-person pose estimation from a single RGB image. In: ICCV2019, pp. 10132–10141 (2019)
Martinez, J., Hossain, R., Romero, J., Little, J.J.: A simple yet effective baseline for 3D human pose estimation. In: ICCV2017, pp. 2659–2668 (2017)
Chiu, H., Adeli, E., Wang, B., Huang, D., Niebles, J.C.: Action-agnostic human pose forecasting. In: 2019 IEEE Winter Conference on Applications of Computer Vision, pp. 1423–1432 (2019)
Wu, J., et al.: Single image 3D interpreter network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 365–382. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_22
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Chao, Y.-W., Yang, J., Price, B., Cohen, S., Deng, J.: Forecasting human dynamics from static images. In: CVPR2017, pp. 3643–3651 (2017)
Horiuchi, Y., Makino, Y., Shinoda, H.: Computational foresight: forecasting human body motion in real-time for reducing delays in interactive system. In: Proceedings of 2017 ACM International Conference on Interactive Surfaces and Spaces, pp. 312–317 (2017)
Acknowledgments
This work was supported by JSPS KAKENHI Grant Number JP17H01801. This work was supported in part by the Kansai University Educational Research Enhancement Fund in 2020 under the title of “Real-World Service Innovation through the AI Robot Challenge.”
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mae, Y. et al. (2021). Real-Time Prediction of Future 3D Pose of Person Using RGB-D Camera for Personalized Services. In: Huang, DS., Jo, KH., Li, J., Gribova, V., Bevilacqua, V. (eds) Intelligent Computing Theories and Application. ICIC 2021. Lecture Notes in Computer Science(), vol 12836. Springer, Cham. https://doi.org/10.1007/978-3-030-84522-3_69
Download citation
DOI: https://doi.org/10.1007/978-3-030-84522-3_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84521-6
Online ISBN: 978-3-030-84522-3
eBook Packages: Computer ScienceComputer Science (R0)