skip to main content
10.1145/3664647.3681582acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

PIMT: Physics-Based Interactive Motion Transition for Hybrid Character Animation

Published: 28 October 2024 Publication History

Abstract

Motion transitions, which serve as bridges between two sequences of character animation, play a crucial role in creating long variable animation for real-time 3D interactive applications. In this paper, we present a framework to produce hybrid character animation, which combines motion capture animation and physical simulation animation that seamlessly connects the front and back motion clips. In contrast to previous works using interpolation for transition, our physics-based approach inherently ensures physical validity, and both the transition moment of the source motion clip and the horizontal rotation of the target motion clip can be specified arbitrarily within a certain range, which achieves high responsiveness and wide latitude for user control. The control policy of character can be trained automatically using only the motion capture data that requires transition, and is enhanced by our proposed Self-Behavior Cloning (SBC), an approach to improve the unsupervised reinforcement learning of motion transition. We show that our framework can accomplish the interactive transition tasks from a fully-connected state machine constructed from nine motion clips with high accuracy and naturalness.

References

[1]
Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba. 2017. Hindsight experience replay. In Advances in Neural Information Processing Systems (Long Beach, CA, United states), Vol. 2017-December. Neural information processing systems foundation, California, CA, USA, 5049 -- 5059.
[2]
Okan Arikan, David A. Forsyth, and James F. O'Brien. 2005. Pushing People Around. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA '05). Association for Computing Machinery, New York, NY, USA, 59--66. https://doi.org/10.1145/1073368.1073376
[3]
Armin Bruderlin and Lance Williams. 1995. Motion Signal Processing. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95). Association for Computing Machinery, New York, NY, USA, 97--104. https://doi.org/10.1145/218380.218421
[4]
Cameron Egbert, Parris K. Egbert, and Bryan S. Morse. 2010. Real-time motion transition by example. In Articulated Motion and Deformable Objects (Port d'Andratx, Mallorca, Spain) (Lecture Notes in Computer Science, Vol. 6169 LNCS). Springer-Verlag Berlin, Heidelberger Platz 3, D-14197 Berlin, Germany, 138 -- 147. http://dx.doi.org/10.1007/978--3--642--14061--7_14
[5]
Epic. 2019. State machines. Epic Games, Inc. https://docs.unrealengine.com/5.3/en-US/state-machines-in-unreal-engine/ Retrieved Nov 27, 2023 from
[6]
Michael Gleicher, Hyun Joon Shin, Lucas Kovar, and Andrew Jepsen. 2003. Snap-Together Motion: Assembling Run-Time Animations. In Proceedings of the 2003 Symposium on Interactive 3D Graphics (Monterey, California) (I3D '03). Association for Computing Machinery, New York, NY, USA, 181--188. https://doi.org/10.1145/641480.641515
[7]
Deepak Gopinath, Hanbyul Joo, and Jungdam Won. 2022. Motion In-Betweening for Physically Simulated Characters. In SIGGRAPH Asia 2022 Posters (Daegu, Republic of Korea) (SA '22). Association for Computing Machinery, New York, NY, USA, Article 7, 2 pages. https://doi.org/10.1145/3550082.3564186
[8]
Félix G. Harvey, Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal. 2020. Robust Motion In-Betweening. ACM Trans. Graph., Vol. 39, 4, Article 60 (aug 2020), 12 pages. https://doi.org/10.1145/3386569.3392480
[9]
Alejandro Hernandez, Jurgen Gall, and Francesc Moreno. 2019. Human motion prediction via spatio-temporal inpainting. In 2019 IEEE/CVF International Conference on Computer Vision (Seoul, South Korea,) (ICCV 2019, Vol. 2019-October). IEEE, New York, NY, USA, 7133 -- 7142. http://dx.doi.org/10.1109/ICCV.2019.00723
[10]
Leslie Ikemoto, Okan Arikan, and David Forsyth. 2007. Quick Transitions with Cached Multi-Way Blends. In Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games (Seattle, Washington) (I3D '07). Association for Computing Machinery, New York, NY, USA, 145--151. https://doi.org/10.1145/1230100.1230125
[11]
Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, and Otmar Hilliges. 2020. Convolutional Autoencoders for Human Motion Infilling. In 2020 International Conference on 3D Vision (3DV 2020). IEEE, New York, NY, USA, 918 -- 927. http://dx.doi.org/10.1109/3DV50981.2020.00102
[12]
Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Trans. Graph., Vol. 21, 3 (jul 2002), 473--482. https://doi.org/10.1145/566654.566605
[13]
Yuki Koyama and Masataka Goto. 2019. Precomputed optimal one-hop motion transition for responsive character animation. Visual Computer, Vol. 35, 6--8 (2019), 1131 -- 1142. http://dx.doi.org/10.1007/s00371-019-01693--8
[14]
Jehee Lee, Jinxiang Chai, Paul S. A. Reitsma, Jessica K. Hodgins, and Nancy S. Pollard. 2002. Interactive Control of Avatars Animated with Human Motion Data. ACM Trans. Graph., Vol. 21, 3 (jul 2002), 491--500. https://doi.org/10.1145/566654.566607
[15]
Jehee Lee and Kang Hoon Lee. 2004. Precomputing Avatar Behavior from Human Motion Data. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Grenoble, France) (SCA '04). Eurographics Association, Goslar, DEU, 79--87. https://doi.org/10.1145/1028523.1028535
[16]
Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable muscle-actuated human simulation and control. ACM Trans. Graph., Vol. 38, 4, Article 73 (jul 2019), 13 pages. https://doi.org/10.1145/3306346.3322972
[17]
Yunhao Li, Zhenbo Yu, Yucheng Zhu, Bingbing Ni, Guangtao Zhai, and Wei Shen. 2022. Skeleton2Humanoid: Animating Simulated Characters for Physically-Plausible Motion In-Betweening. In Proceedings of the 30th ACM International Conference on Multimedia (Lisboa, Portugal) (MM '22). Association for Computing Machinery, New York, NY, USA, 1493--1502. https://doi.org/10.1145/3503161.3548093
[18]
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. 2021. Isaac Gym: High Performance GPU-based physics simulation for robot learning. (2021).
[19]
Lucas Mourot, Ludovic Hoyet, Francois Le Clerc, Francois Schnitzler, and Pierre Hellier. 2022. A Survey on Deep Learning for Skeleton-Based Human Animation. Computer Graphics Forum, Vol. 41, 1 (2022), 122 -- 157. http://dx.doi.org/10.1111/cgf.14426
[20]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In 2019 Advances in Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS 2019, Vol. 32). NIPS, California, USA.
[21]
Xue Bin Peng, Glen Berseth, and Michiel van de Panne. 2016. Terrain-Adaptive Locomotion Skills Using Deep Reinforcement Learning. ACM Trans. Graph., Vol. 35, 4, Article 81 (jul 2016), 12 pages. https://doi.org/10.1145/2897824.2925881
[22]
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. 2022. ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters. ACM Trans. Graph., Vol. 41, 4, Article 94 (jul 2022), 17 pages. https://doi.org/10.1145/3528223.3530110
[23]
Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. 2021. AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control. ACM Trans. Graph., Vol. 40, 4, Article 144 (jul 2021), 20 pages. https://doi.org/10.1145/3450626.3459670
[24]
Ken Perlin. 1995. Real time responsive animation with personality. IEEE transactions on visualization and Computer Graphics, Vol. 1, 1 (1995), 5--15.
[25]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. (2017).
[26]
A. Shapiro, F. Pighin, and P. Faloutsos. 2003. Hybrid control for interactive character animation. In Proceedings of the 11th Pacific Conference on Computer Graphics and Applications (Canmore, AB, Canada), Vol. 2003-January. IEEE COMPUTER SOC, Los Alamitos, CA, USA, 455 -- 461.
[27]
Jie Tan, Karen Liu, and Greg Turk. 2011. Stable proportional-derivative controllers. IEEE Computer Graphics and Applications, Vol. 31, 4 (2011), 34 -- 44. http://dx.doi.org/10.1109/MCG.2011.30
[28]
Xiangjun Tang, He Wang, Bo Hu, Xu Gong, Ruifan Yi, Qilong Kou, and Xiaogang Jin. 2022. Real-Time Controllable Motion Transition for Characters. ACM Trans. Graph., Vol. 41, 4, Article 137 (jul 2022), 10 pages. https://doi.org/10.1145/3528223.3530090
[29]
Xiangjun Tang, Linjun Wu, He Wang, Bo Hu, Xu Gong, Yuchen Liao, Songnan Li, Qilong Kou, and Xiaogang Jin. 2023. RSMT: Real-Time Stylized Motion Transition for Characters. In ACM SIGGRAPH 2023 Conference Proceedings (Los Angeles, CA, USA) (SIGGRAPH '23). Association for Computing Machinery, New York, NY, USA, Article 38, 10 pages. https://doi.org/10.1145/3588432.3591514
[30]
Adrien Treuille, Yongjoon Lee, and Zoran Popović. 2007. Near-Optimal Character Animation with Continuous Control. In ACM SIGGRAPH 2007 Papers (San Diego, California) (SIGGRAPH '07). Association for Computing Machinery, New York, NY, USA, 7--es. https://doi.org/10.1145/1275808.1276386
[31]
Unity. 2018. Unity?manual: State machine basics. Unity Technologies. https://docs.unity3d.com/Manual/StateMachineBasics.html Retrieved Nov 27, 2023 from
[32]
Herwin Van Welbergen, Ben JH Van Basten, Arjan Egges, Zs M Ruttkay, and Mark H Overmars. 2010. Real Time Animation of Virtual Humans: A Trade-off Between Naturalness and Control. COMPUTER GRAPHICS FORUM, Vol. 29, 8 (2010), 2530--2554.
[33]
Jing Wang and Bobby Bodenheimer. 2008. Synthesis and Evaluation of Linear Motion Transitions. ACM Trans. Graph., Vol. 27, 1, Article 1 (mar 2008), 15 pages. https://doi.org/10.1145/1330511.1330512
[34]
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. 2020. A Scalable Approach to Control Diverse Behaviors for Physically Simulated Characters. ACM Trans. Graph., Vol. 39, 4, Article 33 (aug 2020), 12 pages. https://doi.org/10.1145/3386569.3392381
[35]
Wenhao Yu, Greg Turk, and C. Karen Liu. 2018. Learning symmetric and low-energy locomotion. ACM Trans. Graph., Vol. 37, 4, Article 144 (jul 2018), 12 pages. https://doi.org/10.1145/3197517.3201397
[36]
Victor Brian Zordan, Anna Majkowska, Bill Chiu, and Matthew Fast. 2005. Dynamic Response for Motion Capture Animation. In ACM SIGGRAPH 2005 Papers (Los Angeles, California) (SIGGRAPH '05). Association for Computing Machinery, New York, NY, USA, 697--701. https://doi.org/10.1145/1186822.1073249

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '24: Proceedings of the 32nd ACM International Conference on Multimedia
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. character animation
  2. physics-based simulation and control
  3. state machine
  4. unsupervised reinforcement learning

Qualifiers

  • Research-article

Conference

MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia

Acceptance Rates

MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 45
    Total Downloads
  • Downloads (Last 12 months)45
  • Downloads (Last 6 weeks)6
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media