Erwin Wu
Hideki Koike
ABSTRACT
We propose a real-time human motion forecasting system which visualize the future pose in virtual reality using a RGB camera. Our system consists of three parts: 2D pose estimation from RGB frames using a residual neural network, 2D pose forecasting using a recurrent neural network, and 3D recovery from the predicted 2D pose using a residual linear network. To improve the prediction learning quantity of temporal feature, we propose a special method using lattice optical flow for the joints movement estimation. After fitting the skeleton, a predicted 3d model of target human will be built 0.5s in advance in a 30-fps video.
Supplemental Material
- Yu-Wei Chao, Jimei Yang, Brian Price, Scott Cohen, and Jia Deng. 2017. Forecasting human dynamics from static images. In IEEE CVPR.Google Scholar
- Yuuki Horiuchi, Yasutoshi Makino, and Hiroyuki Shinoda. 2017. Computational Foresight: Forecasting Human Body Motion in Real-time for Reducing Delays in Interactive System. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. ACM, 312--317. Google ScholarDigital Library
- Julieta Martinez, Rayat Hossain, Javier Romero, and James J. Little. 2017. A simple yet effective baseline for 3d human pose estimation. In ICCV.Google Scholar
- Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics 36, 4, 14. Google ScholarDigital Library
Index Terms
- Real-time human motion forecasting using a RGB camera
Recommendations
Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera
This article proposes a real-time method that uses a single-view RGB-D input (a depth sensor integrated with a color camera) to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion, and per-...
Outdoor Human Motion Capture by Simultaneous Optimization of Pose and Camera Parameters
We present a method for capturing the skeletal motions of humans using a sparse set of potentially moving cameras in an uncontrolled environment. Our approach is able to track multiple people even in front of cluttered and non-static backgrounds, and ...
Real-time camera motion tracking in planar view scenarios
We propose a novel method for real-time camera motion tracking in planar view scenarios. This method relies on the geometry of a tripod, an initial estimation of camera pose for the first video frame and a primitive tracking procedure. This process uses ...
Comments