Avinash Ranganath
ABSTRACT
Locomotion in humans and animals is highly coordinated, with many joints moving together. Learning similar coordinated locomotion in articulated virtual characters, in the absence of reference motion data, is a challenging task due to the high number of degrees of freedom and the redundancy that comes with it. In this paper, we present a method for learning locomotion for virtual characters in a low dimensional latent space which defines how different joints move together. We introduce a technique called motor babble, wherein a character interacts with its environment by actuating its joints through uncoordinated, low-level (motor) excitations, resulting in a corpus of motion data from which a manifold latent space is extracted. Dimensions of the extracted manifold define a wide variety of synergies pertaining to the character and, through reinforcement learning, we train the character to learn locomotion in the latent space by selecting a small set of appropriate latent dimensions, along with learning the corresponding policy.
Supplemental Material
Available for Download
- Stelian Coros, Andrej Karpathy, Ben Jones, Lionel Reveret, and Michiel Van De Panne. 2011. Locomotion skills for simulated quadrupeds. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1–12.Google Scholar
Digital Library
- Alessandro Crespi, André Badertscher, André Guignard, and Auke Jan Ijspeert. 2005. AmphiBot I: an amphibious snake-like robot. Robotics and Autonomous Systems 50, 4 (2005), 163–175.Google ScholarCross Ref
- Alessandro Crespi and Auke Jan Ijspeert. 2006. AmphiBot II: An amphibious snake robot that crawls and swims using a central pattern generator. In Proceedings of the 9th international conference on climbing and walking robots (CLAWAR 2006). 19–27.Google Scholar
- Alessandro Crespi and Auke Jan Ijspeert. 2008. Online optimization of swimming and crawling in an amphibious snake robot. IEEE Transactions on Robotics 24, 1 (2008), 75–87.Google ScholarDigital Library
- Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and Pieter Abbeel. 2016. Benchmarking deep reinforcement learning for continuous control. In International conference on machine learning. PMLR, 1329–1338.Google Scholar
- Thomas Geijtenbeek and Nicolas Pronost. 2012. Interactive character animation using simulated physics: A state-of-the-art review. In Computer graphics forum, Vol. 31. Wiley Online Library, 2492–2515.Google Scholar
- Radek Grzeszczuk and Demetri Terzopoulos. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. 63–70.Google ScholarDigital Library
- Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning. PMLR, 1861–1870.Google Scholar
- Nikolaus Hansen. 2006. The CMA evolution strategy: a comparing review. Towards a new evolutionary computation(2006), 75–102.Google Scholar
- Nicolas Heess, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, SM Eslami, Martin Riedmiller, 2017. Emergence of locomotion behaviours in rich environments. arXiv preprint arXiv:1707.02286(2017).Google Scholar
- Daniel Holden, Taku Komura, and Jun Saito. 2017. Phase-functioned Neural Networks for Character Control. ACM Transactions on Graphics 36, 4, Article 42(2017), 13 pages.Google ScholarDigital Library
- Fumiya Iida and Rolf Pfeifer. 2004. Cheap rapid locomotion of a quadruped robot: Self-stabilization of bounding gait. In Intelligent autonomous systems, Vol. 8. IOS Press Amsterdam, The Netherlands, 642–649.Google Scholar
- Auke Jan Ijspeert. 2008. Central pattern generators for locomotion control in animals and robots: a review. Neural networks 21, 4 (2008), 642–653.Google ScholarDigital Library
- Auke Jan Ijspeert, Alessandro Crespi, Dimitri Ryczko, and Jean-Marie Cabelguen. 2007. From swimming to walking with a salamander robot driven by a spinal cord model. science 315, 5817 (2007), 1416–1420.Google Scholar
- Sumit Jain and C Karen Liu. 2011. Modal-space control for articulated characters. ACM Transactions on Graphics 30, 5 (2011), 118.Google ScholarDigital Library
- Eunjung Ju, Jungdam Won, Jehee Lee, Byungkuk Choi, Junyong Noh, and Min Gyu Choi. 2013. Data-driven control of flapping flight. ACM Transactions on Graphics (TOG) 32, 5 (2013), 1–12.Google ScholarDigital Library
- Nate Kohl and Peter Stone. 2004. Policy gradient reinforcement learning for fast quadrupedal locomotion. In IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004, Vol. 3. IEEE, 2619–2624.Google ScholarCross Ref
- Paul G Kry, Lionel Revéret, François Faure, and M-P Cani. 2009. Modal locomotion: Animating virtual characters with natural vibrations. In Computer Graphics Forum, Vol. 28. 289–298.Google Scholar
- Yoonsang Lee, Sungeun Kim, and Jehee Lee. 2010. Data-driven biped control. ACM Transactions on Graphics 29, 4 (2010), 129.Google ScholarDigital Library
- Ying-Sheng Luo, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen. 2020. CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion. ACM Transactions on Graphics 39, 4 (2020), 10 pages.Google ScholarDigital Library
- Gavin SP Miller. 1988. The motion dynamics of snakes and worms. In Proceedings of the 15th annual conference on Computer graphics and interactive techniques. 169–173.Google ScholarDigital Library
- Sehee Min, Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. 2019. Softcon: Simulation and control of soft-bodied animals with biomimetic actuators. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–12.Google ScholarDigital Library
- Rubens F Nunes, Joaquim B Cavalcante-Neto, Creto A Vidal, Paul G Kry, and Victor B Zordan. 2012. Using natural vibrations to guide control for locomotion. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 87–94.Google ScholarDigital Library
- Tony Owen. 1994. Biologically Inspired Robots: Snake-Like Locomotors and Manipulators by Shigeo Hirose Oxford University Press, Oxford, 1993, 220pages, incl. index (£ 40). Robotica 12, 3 (1994), 282–282.Google Scholar
- Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and Jehee Lee. 2019. Learning Predict-and-Simulate Policies From Unorganized Human Motion Data. ACM Transactions on Graphics 38, 6, Article 205(2019).Google ScholarDigital Library
- Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018a. DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. ACM Transactions on Graphics(2018).Google Scholar
- Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2016. Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning. ACM Transactions on Graphics (Proc. SIGGRAPH 2016) 35, 4 (2016).Google Scholar
- Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics 36, 4 (2017), 41.Google ScholarDigital Library
- Xue Bin Peng, Michael Chang, Grace Zhang, Pieter Abbeel, and Sergey Levine. 2019. MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies. In Annual Conference on Neural Information Processing Systems, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 3681–3692.Google Scholar
- Xue Bin Peng, Erwin Coumans, Tingnan Zhang, Tsang-Wei Edward Lee, Jie Tan, and Sergey Levine. 2020. Learning Agile Robotic Locomotion Skills by Imitating Animals. In Robotics: Science and Systems. https://doi.org/10.15607/RSS.2020.XVI.064Google Scholar
- Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, and Sergey Levine. 2018b. SFV: Reinforcement Learning of Physical Skills from Videos. ACM Transactions on Graphics(2018).Google ScholarDigital Library
- Marc H Raibert. 1986. Legged robots. Commun. ACM 29, 6 (1986), 499–514.Google ScholarDigital Library
- Marc H Raibert and Jessica K Hodgins. 1991. Animation of dynamic legged locomotion. In Proceedings of the 18th annual conference on Computer graphics and interactive techniques. 349–358.Google ScholarDigital Library
- Avinash Ranganath, Pei Xu, Ioannis Karamouzas, and Victor Zordan. 2019. Low Dimensional Motor Skill Learning Using Coactivation. In Motion, Interaction and Games. 1–10.Google Scholar
- Ryo Saegusa, Giorgio Metta, and Giulio Sandini. 2009. Active learning for multiple sensorimotor coordination based on state confidence. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2598–2603.Google ScholarCross Ref
- John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438(2015).Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017a. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347(2017).Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017b. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347(2017).Google Scholar
- Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating Biped Behaviors from Human Motion Data. ACM Transactions on Graphics 26, 3 (2007), 107–es. https://doi.org/10.1145/1276377.1276511Google ScholarDigital Library
- Jie Tan, Yuting Gu, Greg Turk, and C Karen Liu. 2011. Articulated swimming creatures. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1–12.Google ScholarDigital Library
- Michiel Van De Panne. 1996. Parameterized gait synthesis. IEEE Computer Graphics and Applications 16, 2 (1996), 40–49.Google ScholarDigital Library
- Michiel Van de Panne, Ryan Kim, and Eugene Fiume. 1994. Virtual wind-up toys for animation. In Graphics Interface. Citeseer, 208–208.Google Scholar
- Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Transactions on Graphics (TOG) 28, 3 (2009), 1–8.Google ScholarDigital Library
- Kevin Wampler, Zoran Popović, and Jovan Popović. 2014. Generalizing locomotion style to new animals with inverse optimal regression. ACM Transactions on Graphics 33, 4 (2014), 1–11.Google ScholarDigital Library
- Jungdam Won, Deepak Gopinath, and Jessica Hodgins. 2020. A scalable approach to control diverse behaviors for physically simulated characters. ACM Transactions on Graphics 39, 4 (2020), 33–1.Google ScholarDigital Library
- Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to train your dragon: example-guided control of flapping flight. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1–13.Google ScholarDigital Library
- Jungdam Won, Jungnam Park, and Jehee Lee. 2018. Aerobatics control of flying creatures via self-regulated learning. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–10.Google ScholarDigital Library
- Jia-chi Wu and Zoran Popović. 2003. Realistic modeling of bird flight animations. ACM Transactions on Graphics (TOG) 22, 3 (2003), 888–895.Google ScholarDigital Library
- Wenhao Yu, Greg Turk, and C Karen Liu. 2018. Learning symmetric and low-energy locomotion. ACM Transactions on Graphics 37, 4 (2018), 144.Google ScholarDigital Library
Recommendations
PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network
MIG '21: Proceedings of the 14th ACM SIGGRAPH Conference on Motion, Interaction and GamesData-driven methods for physics-based character control using reinforcement learning have been successfully applied to generate high-quality motions. However, existing approaches typically rely on Gaussian distributions to represent the action policy, ...
SoftCon: simulation and control of soft-bodied animals with biomimetic actuators
We present a novel and general framework for the design and control of underwater soft-bodied animals. The whole body of an animal consisting of soft tissues is modeled by tetrahedral and triangular FEM meshes. The contraction of muscles embedded in the ...
Composite Motion Learning with Task Control
We present a deep learning method for composite and task-driven motion control for physically simulated characters. In contrast to existing data-driven approaches using reinforcement learning that imitate full-body motions, we learn decoupled motions for ...
Comments