skip to main content
10.1145/3641519.3657437acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article
Open access

Strategy and Skill Learning for Physics-based Table Tennis Animation

Published: 13 July 2024 Publication History

Abstract

Recent advancements in physics-based character animation leverage deep learning to generate agile and natural motion, enabling characters to execute movements such as backflips, boxing, and tennis. However, reproducing the selection and use of diverse motor skills in dynamic environments to solve complex tasks, as humans do, still remains a challenge. We present a strategy and skill learning approach for physics-based table tennis animation. Our method addresses the issue of mode collapse, where the characters do not fully utilize the motor skills they need to perform to execute complex tasks. More specifically, we demonstrate a hierarchical control system for diversified skill learning and a strategy learning framework for effective decision-making. We showcase the efficacy of our method through comparative analysis with state-of-the-art methods, demonstrating its capabilities in executing various skills for table tennis. Our strategy learning framework is validated through both agent-agent interaction and human-agent interaction in Virtual Reality, handling both competitive and cooperative tasks.

Supplemental Material

MP4 File - presentation
presentation
MP4 File
Supplementary video in "video.mp4" and appendix in "appendix.pdf".

References

[1]
Jinseok Bae, Jungdam Won, Donggeun Lim, Cheol-Hui Min, and Young Min Kim. 2023. PMP: Learning to Physically Interact with Environments using Part-wise Motion Priors. In ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH 2023. ACM, 64:1–64:10. https://doi.org/10.1145/3588432.3591487
[2]
Akhil Bagaria and George Konidaris. 2020. Option Discovery using Deep Skill Chaining. In 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net. https://openreview.net/forum?id=B1gqipNYwH
[3]
Kevin Bergamin, Simon Clavet, Daniel Holden, and James Richard Forbes. 2019. DReCon: data-driven responsive control of physics-based characters. ACM Trans. Graph. 38, 6 (2019), 206:1–206:11. https://doi.org/10.1145/3355089.3356536
[4]
Dylan J. A. Brenneis, Adam S. R. Parker, Michael Bradley Johanson, Andrew Butcher, Elnaz Davoodi, Leslie Acker, Matthew M. Botvinick, Joseph Modayil, Adam White, and Patrick M. Pilarski. 2021. Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study. arXiv preprint arXiv:2112.07774 (2021). https://arxiv.org/abs/2112.07774
[5]
Xuxin Cheng, Ashish Kumar, and Deepak Pathak. 2023. Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion. In IEEE International Conference on Robotics and Automation, ICRA 2023. IEEE, 5106–5112. https://doi.org/10.1109/ICRA48891.2023.10161470
[6]
Martin de Lasa, Igor Mordatch, and Aaron Hertzmann. 2010. Feature-based locomotion controllers. ACM Trans. Graph. 29, 4 (2010), 131:1–131:10. https://doi.org/10.1145/1778765.1781157
[7]
Zhiyang Dou, Xuelin Chen, Qingnan Fan, Taku Komura, and Wenping Wang. 2023. C · ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters. In SIGGRAPH Asia 2023 Conference Papers, SA 2023. ACM, 2:1–2:11. https://doi.org/10.1145/3610548.3618205
[8]
Kevin French, Shiyu Wu, Tianyang Pan, Zheming Zhou, and Odest Chadwicke Jenkins. 2019. Learning Behavior Trees From Demonstration. In International Conference on Robotics and Automation, ICRA 2019. IEEE, 7791–7797. https://doi.org/10.1109/ICRA.2019.8794104
[9]
Jessica K. Hodgins, Wayne L. Wooten, David C. Brogan, and James F. O’Brien. 1995. Animating Human Athletics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1995. ACM, 71–78. https://doi.org/10.1145/218380.218414
[10]
Arushi Jain, Khimya Khetarpal, and Doina Precup. 2021. Safe option-critic: learning safety in the option-critic architecture. Knowl. Eng. Rev. 36 (2021), e4. https://doi.org/10.1017/S0269888921000035
[11]
Martin Klissarov, Pierre-Luc Bacon, Jean Harb, and Doina Precup. 2017. Learnings Options End-to-End for Continuous Action Tasks. arXiv preprint arXiv:1712.00004 (2017). http://arxiv.org/abs/1712.00004
[12]
George Dimitri Konidaris and Andrew G. Barto. 2009. Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining. In Advances in Neural Information Processing Systems 22. 1015–1023. https://proceedings.neurips.cc/paper/2009/hash/e0cf1f47118daebc5b16269099ad7347-Abstract.html
[13]
Taesoo Kwon, Young-Sang Cho, Sang Il Park, and Sung Yong Shin. 2008. Two-Character Motion Analysis and Synthesis. IEEE Trans. Vis. Comput. Graph. 14, 3 (2008), 707–720. https://doi.org/10.1109/TVCG.2008.22
[14]
Joseph Laszlo, Michiel van de Panne, and Eugene Fiume. 1996. Limit Cycle Control and Its Application to the Animation of Balancing and Walking. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1996. ACM, 155–162. https://doi.org/10.1145/237170.237231
[15]
Youngwoon Lee, Joseph J. Lim, Anima Anandkumar, and Yuke Zhu. 2021. Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization. In Conference on Robot Learning, 2021(Proceedings of Machine Learning Research, Vol. 164). PMLR, 406–416. https://proceedings.mlr.press/v164/lee22a.html
[16]
Youngwoon Lee, Shao-Hua Sun, Sriram Somasundaram, Edward S. Hu, and Joseph J. Lim. 2019. Composing Complex Skills by Learning Transition Policies. In 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net. https://openreview.net/forum?id=rygrBhC5tQ
[17]
Chengxi Li, Pai Zheng, Shufei Li, Yat Ming Pang, and Carman K. M. Lee. 2022. AR-assisted digital twin-enabled robot collaborative manufacturing system with human-in-the-loop. Robotics Comput. Integr. Manuf. 76 (2022), 102321. https://doi.org/10.1016/J.RCIM.2022.102321
[18]
C. Karen Liu, Aaron Hertzmann, and Zoran Popovic. 2006. Composition of complex optimal multi-character motions. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 2006. Eurographics Association, 215–222. https://doi.org/10.2312/SCA/SCA06/215-222
[19]
Huimin Liu, Zhiquan Wang, Christos Mousas, and Dominic Kao. 2020. Virtual Reality Racket Sports: Virtual Drills for Exercise and Training. In 2020 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020. IEEE, 566–576. https://doi.org/10.1109/ISMAR50242.2020.00084
[20]
Libin Liu and Jessica K. Hodgins. 2017. Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q-Learning. ACM Trans. Graph. 36, 3 (2017), 29:1–29:14. https://doi.org/10.1145/3083723
[21]
Libin Liu, Michiel van de Panne, and KangKang Yin. 2016. Guided Learning of Control Graphs for Physics-Based Characters. ACM Trans. Graph. 35, 3 (2016), 29:1–29:14. https://doi.org/10.1145/2893476
[22]
Libin Liu, KangKang Yin, Michiel van de Panne, Tianjia Shao, and Weiwei Xu. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph. 29, 4 (2010), 128:1–128:10. https://doi.org/10.1145/1778765.1778865
[23]
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. 2021. Isaac Gym: High Performance GPU Based Physics Simulation For Robot Learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). https://openreview.net/forum?id=fgFBtYgJQX_
[24]
Alejandro Marzinotto, Michele Colledanchise, Christian Smith, and Petter Ögren. 2014. Towards a unified behavior trees framework for robot control. In 2014 IEEE International Conference on Robotics and Automation, ICRA 2014. IEEE, 5420–5427. https://doi.org/10.1109/ICRA.2014.6907656
[25]
Josh Merel, Leonard Hasenclever, Alexandre Galashov, Arun Ahuja, Vu Pham, Greg Wayne, Yee Whye Teh, and Nicolas Heess. 2019. Neural Probabilistic Motor Primitives for Humanoid Control. In 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net. https://openreview.net/forum?id=BJl6TjRcY7
[26]
Igor Mordatch, Emanuel Todorov, and Zoran Popovic. 2012. Discovery of complex behaviors through contact-invariant optimization. ACM Trans. Graph. 31, 4 (2012), 43:1–43:8. https://doi.org/10.1145/2185520.2185539
[27]
Junhyuk Oh, Yijie Guo, Satinder Singh, and Honglak Lee. 2018. Self-Imitation Learning. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018(Proceedings of Machine Learning Research, Vol. 80). PMLR, 3875–3884. http://proceedings.mlr.press/v80/oh18b.html
[28]
Stefan Pastel, Katharina Petri, C. H. Chen, Ana Milena Wiegand Cáceres, M. Stirnatis, C. Nübel, Lasse Schlotter, and Kerstin Witte. 2023. Training in virtual reality enables learning of a complex sports movement. Virtual Real. 27, 2 (2023), 523–540. https://doi.org/10.1007/S10055-022-00679-7
[29]
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. DeepMimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph. 37, 4 (2018), 143. https://doi.org/10.1145/3197517.3201311
[30]
Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel van de Panne. 2017. DeepLoco: dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Trans. Graph. 36, 4 (2017), 41:1–41:13. https://doi.org/10.1145/3072959.3073602
[31]
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. 2022. ASE: large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Trans. Graph. 41, 4 (2022), 94:1–94:17. https://doi.org/10.1145/3528223.3530110
[32]
Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. 2021. AMP: adversarial motion priors for stylized physics-based character control. ACM Trans. Graph. 40, 4 (2021), 144:1–144:20. https://doi.org/10.1145/3450626.3459670
[33]
Mingyo Seo, Steve Han, Kyutae Sim, SeungHyeon Bang, Carlos Gonzalez, Luis Sentis, and Yuke Zhu. 2023. Deep Imitation Learning for Humanoid Loco-manipulation Through Human Teleoperation. In 22nd IEEE-RAS International Conference on Humanoid Robots, Humanoids 2023. IEEE, 1–8. https://doi.org/10.1109/HUMANOIDS57100.2023.10375203
[34]
Hubert P. H. Shum, Taku Komura, Masashi Shiraishi, and Shuntaro Yamazaki. 2008. Interaction patches for multi-character animation. ACM Trans. Graph. 27, 5 (2008), 114. https://doi.org/10.1145/1409060.1409067
[35]
Hubert Pak Ho Shum, Taku Komura, and Shuntaro Yamazaki. 2012. Simulating Multiple Character Interactions with Collaborative and Adversarial Goals. IEEE Trans. Vis. Comput. Graph. 18, 5 (2012), 741–752. https://doi.org/10.1109/TVCG.2010.257
[36]
Richard S. Sutton, Doina Precup, and Satinder Singh. 1999. Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Artif. Intell. 112, 1-2 (1999), 181–211. https://doi.org/10.1016/S0004-3702(99)00052-1
[37]
Eleven Table Tennis. 2016. https://elevenvr.com
[38]
Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng. 2023. CALM: Conditional Adversarial Latent Models for Directable Virtual Characters. In ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH 2023. ACM, 37:1–37:9. https://doi.org/10.1145/3588432.3591541
[39]
Kevin Wampler, Erik Andersen, Evan Herbst, Yongjoon Lee, and Zoran Popovic. 2010. Character animation in two-player adversarial games. ACM Trans. Graph. 29, 3 (2010), 26:1–26:13. https://doi.org/10.1145/1805964.1805970
[40]
Chao Wang, Anna Belardinelli, Stephan Hasler, Theodoros Stouraitis, Daniel Tanneberg, and Michael Gienger. 2023. Explainable Human-Robot Training and Cooperation with Augmented Reality. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA 2023. ACM, 449:1–449:5. https://doi.org/10.1145/3544549.3583889
[41]
Jungdam Won, Deepak Gopinath, and Jessica K. Hodgins. 2020. A scalable approach to control diverse behaviors for physically simulated characters. ACM Trans. Graph. 39, 4 (2020), 33. https://doi.org/10.1145/3386569.3392381
[42]
Jungdam Won, Deepak Gopinath, and Jessica K. Hodgins. 2021. Control strategies for physically simulated characters performing two-player competitive sports. ACM Trans. Graph. 40, 4 (2021), 146:1–146:11. https://doi.org/10.1145/3450626.3459761
[43]
Jungdam Won, Deepak Gopinath, and Jessica K. Hodgins. 2022. Physics-based character controllers using conditional VAEs. ACM Trans. Graph. 41, 4 (2022), 96:1–96:12. https://doi.org/10.1145/3528223.3530067
[44]
Pei Xu, Xiumin Shang, Victor B. Zordan, and Ioannis Karamouzas. 2023. Composite Motion Learning with Task Control. ACM Trans. Graph. 42, 4 (2023), 93:1–93:16. https://doi.org/10.1145/3592447
[45]
Heyuan Yao, Zhenhua Song, Baoquan Chen, and Libin Liu. 2022. ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters. ACM Trans. Graph. 41, 6 (2022), 183:1–183:16. https://doi.org/10.1145/3550454.3555434
[46]
KangKang Yin, Stelian Coros, Philippe Beaudoin, and Michiel van de Panne. 2008. Continuation methods for adapting simulated skills. ACM Trans. Graph. 27, 3 (2008), 81. https://doi.org/10.1145/1360612.1360680
[47]
Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Xue Bin Peng, and Kayvon Fatahalian. 2023. Learning Physically Simulated Tennis Skills from Broadcast Videos. ACM Trans. Graph. 42, 4 (2023), 95:1–95:14. https://doi.org/10.1145/3592408
[48]
Qingxu Zhu, He Zhang, Mengting Lan, and Lei Han. 2023. Neural Categorical Priors for Physics-Based Character Control. ACM Trans. Graph. 42, 6 (2023), 178:1–178:16. https://doi.org/10.1145/3618397

Cited By

View all
  • (2025)Temporal goal-aware transformer assisted visual reinforcement learning for virtual table tennis agentThe Visual Computer10.1007/s00371-025-03822-yOnline publication date: 13-Feb-2025

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '24: ACM SIGGRAPH 2024 Conference Papers
July 2024
1106 pages
ISBN:9798400705250
DOI:10.1145/3641519
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2024

Check for updates

Author Tags

  1. Character Animation
  2. Deep Reinforcement Learning
  3. Multi-character Interaction
  4. Physics-based Characters
  5. Table Tennis
  6. Virtual Reality

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SIGGRAPH '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)749
  • Downloads (Last 6 weeks)121
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Temporal goal-aware transformer assisted visual reinforcement learning for virtual table tennis agentThe Visual Computer10.1007/s00371-025-03822-yOnline publication date: 13-Feb-2025

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media