Kai Ding
Libin Liu
Michiel van de Panne
ABSTRACT
We introduce a method for learning low-dimensional linear feedback strategies for the control of physics-based animated characters around a given reference trajectory. This allows for learned low-dimensional state abstractions and action abstractions, thereby reducing the need to rely on manually designed abstractions such as the center-of-mass state or foot-placement actions. Once learned, the compact feedback structure allow simulated characters to respond to changes in the environment and changes in goals. The approach is based on policy search in the space of reduced-order linear output feedback matrices. We show that these can be used to replace or further reduce manually-designed state and action abstractions. The approach is sufficiently general to allow for the development of unconventional feedback loops, such as feedback based on ground reaction forces. Results are demonstrated for a mix of 2D and 3D systems, including tilting-platform balancing, walking, running, rolling, targeted kicks, and several types of ball-hitting tasks.
Supplemental Material
Available for Download
- Barbič, J., da Silva, M., and Popović, J. 2009. Deformable object animation using reduced optimal control. In ACM SIGGRAPH 2009 papers, ACM, SIGGRAPH '09, 53:1--53:9. Google Scholar
Digital Library
- Berniker, M., Jarc, A., Bizzi, E., and Tresch, M. C. 2009. Simplified and effective motor control based on muscle synergies to exploit musculoskeletal dynamics. Proceedings of the National Academy of Sciences 106, 18, 7601--7606.Google ScholarCross Ref
- Box2D., http://box2d.org/.Google Scholar
- Burke, J., Lewis, A., and Overton, M. 2003. A nonsmooth, nonconvex optimization approach to robust stabilization by static output feedback and low-order controllers. In Proc. ROCOND 2003.Google Scholar
- Burns, J. A., and King, B. B. 1998. A reduced basis approach to the design of low-order feedback controllers for nonlinear continuous systems. Journal of Vibration and Control 4, 3, 297--323.Google ScholarCross Ref
- Coros, S., Beaudoin, P., and van de Panne, M. 2010. Generalized biped walking control. ACM Transctions on Graphics 29, 4, Article 130. Google ScholarDigital Library
- da Silva, M., Abe, Y., and Popović, J. 2008. Interactive simulation of stylized human locomotion. ACM Trans. Graph. 27 (August), 82:1--82:10. Google ScholarDigital Library
- David, J., and De Moor, B. 1994. Designing reduced order output feedback controllers using a potential reduction method. American Control Conference, 1994 1, 845--849.Google ScholarCross Ref
- de Lasa, M., Mordatch, I., and Hertzmann, A. 2010. Feature-Based Locomotion Controllers. ACM Transactions on Graphics 29, 3. Google ScholarDigital Library
- de Oliveira, M., and Geromel, J. 1997. Numerical comparison of output feedback design methods. In American Control Conference, 1997. Proceedings of the 1997, vol. 1, 72--76 vol.1.Google Scholar
- Hansen, N. 2006. The CMA evolution strategy: a comparing review. In Towards a new evolutionary computation. Advances on estimation of distribution algorithms, J. Lozano, P. Larranaga, I. Inza, and E. Bengoetxea, Eds. Springer, 75--102.Google Scholar
- Jain, S., and Liu, C. K. 2011. Modal-space control for articulated characters. ACM Trans. Graph. 30 (October), 118:1--118:12. Google ScholarDigital Library
- James, D. L., and Pai, D. K. 2002. Dyrt: dynamic response textures for real time deformation simulation with graphics hardware. ACM Trans. Graph. 21 (July), 582--585. Google ScholarDigital Library
- Kry, P. G., Reveret, L., Faure, F., and Cani, M.-P. 2009. Modal locomotion: Animating virtual characters with natural vibrations. Computer Grahics Forum 28, 2.Google ScholarCross Ref
- Kwon, T., and Hodgins, J. K. 2010. Control systems for human running using an inverted pendulum model and a reference motion capture sequence. The ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2010). Google ScholarDigital Library
- Lall, S., Marsden, J. E., and Glavaški, S. 2002. A subspace approach to balanced truncation for model reduction of nonlinear control systems. International journal of robust and nonlinear control 12, 6, 519--535.Google Scholar
- Laszlo, J., van de Panne, M., and Eugene, F. 1996. Limit cycle control and its application to the animation of balancing and walking. In SIGGRAPH '96, ACM, 155--162. Google ScholarDigital Library
- Lee, Y., Kim, S., and Lee, J. 2010. Data-driven biped control. ACM Trans. Graph. 29 (July), 129:1--129:8. Google ScholarDigital Library
- Lewis, F., and Syrmos, V. 1995. Optimal control. A Wiley-Interscience publication. J. Wiley.Google Scholar
- Liu, L., Yin, K., van de Panne, M., Shao, T., and Xu, W. 2010. Sampling-based contact-rich motion control. ACM Transctions on Graphics 29, 4, Article 128. Google ScholarDigital Library
- Liu, L., Yin, K., van de Panne, M., and Guo, B. 2012. Terrain runner: control, parameterization, composition, and planning for highly dynamic motions. ACM Trans. Graph. 31, 6, 154. Google ScholarDigital Library
- Liu, L., Yin, K., Wang, B., and Guo, B. 2013. Simulation and control of skeleton-driven soft body characters. ACM Transactions on Graphics (TOG) 32, 6, 215. Google ScholarDigital Library
- Macchietto, A., Zordan, V., and Shelton, C. R. 2009. Momentum control for balance. ACM Trans. Graph. 28 (July), 80:1--80:8. Google ScholarDigital Library
- Mordatch, I., de Lasa, M., and Hertzmann, A. 2010. Robust Physics-Based Locomotion Using Low-Dimensional Planning. ACM Transactions on Graphics 29, 3. Google ScholarDigital Library
- Muico, U., Lee, Y., Popović, J., and Popović, Z. 2009. Contact-aware nonlinear control of dynamic characters. ACM Transactions on Graphics 28, 3. Google ScholarDigital Library
- ODE. Open dynamics engine, http://www.ode.org/.Google Scholar
- Raibert, M. H., and Hodgins, J. K. 1991. Animation of dynamic legged locomotion. SIGGRAPH Comput. Graph. 25 (July), 349--358. Google ScholarDigital Library
- Safonova, A., Hodgins, J. K., and Pollard, N. S. 2004. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23 (Aug.), 514--521. Google ScholarDigital Library
- Scherer, C., Gahinet, P., and Chilali, M. 1997. Multiobjective output-feedback control via lmi optimization. Automatic Control, IEEE Transactions on 42, 7, 896--911.Google ScholarCross Ref
- Slotine, J.-J. E., Li, W., et al. 1991. Applied nonlinear control, vol. 60. Prentice-Hall Englewood Cliffs, NJ.Google Scholar
- Sok, K. W., Kim, M., and Lee, J. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26 (July). Google ScholarDigital Library
- Tan, J., Gu, Y., Liu, C. K., and Turk, G. 2014. Learning bicycle stunts. ACM Transactions on Graphics (TOG) 33, 4, 50. Google ScholarDigital Library
- Treuille, A., Lewis, A., and Popović, Z. 2006. Model reduction for real-time fluids. ACM Transactions on Graphics 25, 3 (July), 826--834. Google ScholarDigital Library
- Tsai, Y.-Y., Lin, W.-C., Cheng, K. B., Lee, J., and Lee, T.-Y. 2010. Real-time physics-based 3d biped character animation using an inverted pendulum model. IEEE Transactions on Visualization and Computer Graphics 16 (March), 325--337. Google ScholarDigital Library
- Wang, J. M., Fleet, D. J., and Hertzmann, A. 2009. Optimizing walking controllers. ACM Trans. Graph. 28 (December), 168:1--168:8. Google ScholarDigital Library
- Wang, J. M., Fleet, D. J., and Hertzmann, A. 2010. Optimizing walking controllers for uncertain inputs and environments. ACM Trans. Graph. 29 (July), 73:1--73:8. Google ScholarDigital Library
- Ye, Y., and Liu, C. K. 2010. Optimal feedback control for character animation using an abstract model. ACM Trans. Graph. 29 (July), 74:1--74:9. Google ScholarDigital Library
- Yin, K., Loken, K., and van de Panne, M. 2007. Simbicon: Simple biped locomotion control. ACM Trans. Graph. 26, 3, Article 105. Google ScholarDigital Library
Index Terms
- Learning reduced-order feedback policies for motion skills
Recommendations
Reduced-order observer-based output feedback control of nonlinear time-delay systems with prescribed performance
This paper studies the problem of output feedback control for a class of nonlinear time-delay systems with prescribed performance. The system is in the form of triangular structure with unmodelled dynamics. First, we introduce a reduced-order observer ...
Partial-state feedback multivariable MRAC and reduced-order designs
AbstractThis paper develops a new model reference adaptive control (MRAC) framework using partial-state feedback for solving a multivariable adaptive output tracking problem. The developed MRAC scheme has full capability to deal with plant ...
Real-time motion retargeting to highly varied user-created morphologies
Character animation in video games---whether manually keyframed or motion captured---has traditionally relied on codifying skeletons early in a game's development, and creating animations rigidly tied to these fixed skeleton morphologies. This paper ...
Comments