skip to main content
research-article

Inverse-Foley animation: synchronizing rigid-body motions to sound

Published: 27 July 2014 Publication History

Abstract

In this paper, we introduce Inverse-Foley Animation, a technique for optimizing rigid-body animations so that contact events are synchronized with input sound events. A precomputed database of randomly sampled rigid-body contact events is used to build a contact-event graph, which can be searched to determine a plausible sequence of contact events synchronized with the input sound's events. To more easily find motions with matching contact times, we allow transitions between simulated contact events using a motion blending formulation based on modified contact impulses. We fine tune synchronization by slightly retiming ballistic motions. Given a sound, our system can synthesize synchronized motions using graphs built with hundreds of thousands of precomputed motions, and millions of contact events. Our system is easy to use, and has been used to plan motions for hundreds of sounds, and dozens of rigid-body models.

Supplementary Material

ZIP File (a41-langlois.zip)
Supplemental material.
MP4 File (a41-sidebyside.mp4)

References

[1]
Arikan, O., and Forsyth, D. A. 2002. Interactive motion generation from examples. ACM Transactions on Graphics 21, 3 (July), 483--490.
[2]
Barzel, R., Hughes, J. F., and Wood, D. N. 1996. Plausible motion simulation for computer graphics animation. In EGCAS 96: Seventh International Workshop on Computer Animation and Simulation, 183--197.
[3]
Belta, C., and Kumar, V. 2002. An SVD-based projection method for interpolation on SE(3). Robotics and Automation, IEEE Transactions on 18, 3, 334--345.
[4]
Bender, J., Erleben, K., and Trinkle, J. 2013. Interactive simulation of rigid body dynamics in computer graphics. In Computer Graphics Forum, Wiley Online Library.
[5]
Bhat, K. S., Seitz, S. M., Popović, J., and Khosla, P. K. 2002. Computing the physical parameters of rigid-body motion from video. In Computer VisionECCV 2002. Springer, 551--565.
[6]
Bruderlin, A., and Williams, L. 1995. Motion signal processing. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 97--104.
[7]
Cardle, M., Barthe, L., Brooks, S., and Robinson, P. 2002. Music-driven motion editing: local motion transformations guided by music analysis. Proceedings 20th Eurographics UK Conference, 38--44.
[8]
Chadwick, J. N., An, S. S., and James, D. L. 2009. Harmonic Shells: A Practical Nonlinear Sound Model for Near-Rigid Thin Shells. ACM Transactions on Graphics 28, 5 (Dec.), 119:1--119:10.
[9]
Chadwick, J. N., Zheng, C., and James, D. L. 2012. Precomputed acceleration noise for improved rigid-body sound. ACM Transactions on Graphics 31, 4 (July), 103:1--103:9.
[10]
Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. Proceedings of the 27th annual conference on, 219--228.
[11]
Chion, M. 1994. Audio-Vision: Sound on Screen. Columbia University Press.
[12]
Cohen, M. 1992. Interactive spacetime control for animation. In ACM SIGGRAPH Computer Graphics, ACM, vol. 26, 293--302.
[13]
Duff, D. J., Morwald, T., Stolkin, R., and Wyatt, J. 2011. Physical simulation for monocular 3d model based tracking. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 5218--5225.
[14]
Hofer, M., and Pottmann, H. 2004. Energy-minimizing splines in manifolds. ACM Transactions on Graphics 23, 3 (Aug.), 284--293.
[15]
Kim, T.-h., Park, S. I., and Shin, S. Y. 2003. Rhythmicmotion synthesis based on motion-beat analysis. ACM Transactions on Graphics 22, 3, 392--401.
[16]
Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3, 473--482.
[17]
Kuffner, J. 2004. Effective sampling and distance metrics for 3D rigid body path planning. In Robotics and Automation, 2004. Proceedings. ICRA'04. 2004 IEEE International Conference on, vol. 4, IEEE, 3993--3998.
[18]
Lee, H.-C., and Lee, I.-K. 2005. Automatic synchronization of background music and motion in computer animation. Computer Graphics Forum 24, 3, 353--361.
[19]
Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Transactions on Graphics 21, 3 (July), 491--500.
[20]
O'Brien, J. F., Cook, P. R., and Essl, G. 2001. Synthesizing sounds from physically based motion. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 529--536.
[21]
O'Sullivan, C., Dingliana, J., Giang, T., and Kaiser, M. K. 2003. Evaluating the visual fidelity of physically based animations. ACM Transactions on Graphics 22, 3 (July) (July), 527--536.
[22]
Popović, J., Seitz, S., Erdmann, M., Popović, Z., and Witkin, A. 2000. Interactive manipulation of rigid body simulations. In Proceedings of ACM SIGGRAPH 2000, ACM Press/Addison-Wesley Publishing Co., 209--218.
[23]
Popović, J., Seitz, S. M., and Erdmann, M. 2003. Motion sketching for control of rigid-body simulations. ACM Transactions on Graphics 22, 4 (Oct.), 1034--1054.
[24]
Shin, H. J., Lee, J., Shin, S. Y., and Gleicher, M. 2001. Computer puppetry: An importance-based approach. ACM Trans. Graph. 20, 2 (Apr.), 67--94.
[25]
Takala, T., and Hahn, J. 1992. Sound rendering. In Computer Graphics (Proceedings of SIGGRAPH 92), 211--220.
[26]
Tang, D., Ngo, J. T., and Marks, J. 1995. N-body spacetime constraints. The Journal of Visualization and Computer Animation 6, 3 (July-Sept.), 143--154.
[27]
Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM Transactions on Graphics 26, 3 (July), 14:1--14:8.
[28]
Twigg, C. D., and James, D. L. 2008. Backward steps in rigid body simulation. ACM Transactions on Graphics 27, 3 (Aug.), 25:1--25:10.
[29]
van den Doel, K., Kry, P. G., and Pai, D. K. 2001. Foleyautomatic: Physically-based sound effects for interactive simulation and animation. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 537--544.
[30]
Witkin, A., and Kass, M. 1988. Spacetime constraints. In Computer Graphics (Proceedings of ACM SIGGRAPH '88), ACM, vol. 22, 159--168.
[31]
Yilmaz, A., Javed, O., and Shah, M. 2006. Object tracking: A survey. ACM Computing Surveys 38, 4, 13.
[32]
Zheng, C., and James, D. L. 2010. Rigid-Body Fracture Sound with Precomputed Soundbanks. ACM Transactions on Graphics 29, 4 (July), 69:1--69:13.

Cited By

View all
  • (2024)Audience perceptions of Foley footsteps and 3D realism designed to convey walker characteristicsPersonal and Ubiquitous Computing10.1007/s00779-024-01819-328:5(779-799)Online publication date: 11-Jun-2024
  • (2023)ViCMA: Visual Control of Multibody AnimationsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618223(1-11)Online publication date: 10-Dec-2023
  • (2023)Rhythm is a Dancer: Music-Driven Motion Synthesis With Global StructureIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316367629:8(3519-3534)Online publication date: 1-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 33, Issue 4
July 2014
1366 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/2601097
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2014
Published in TOG Volume 33, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. motion control
  2. motion graphs
  3. rigid-body dynamics
  4. sound design
  5. sound rendering
  6. sound synthesis

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)28
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Audience perceptions of Foley footsteps and 3D realism designed to convey walker characteristicsPersonal and Ubiquitous Computing10.1007/s00779-024-01819-328:5(779-799)Online publication date: 11-Jun-2024
  • (2023)ViCMA: Visual Control of Multibody AnimationsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618223(1-11)Online publication date: 10-Dec-2023
  • (2023)Rhythm is a Dancer: Music-Driven Motion Synthesis With Global StructureIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316367629:8(3519-3534)Online publication date: 1-Aug-2023
  • (2023)Conditional Generation of Audio from Video via Foley Analogies2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00240(2426-2436)Online publication date: Jun-2023
  • (2022)Manipulating Foley Footsteps and Character Realism to Influence Audience Perceptions of a 3D Animated Walk CycleProceedings of the 17th International Audio Mostly Conference10.1145/3561212.3561221(113-120)Online publication date: 6-Sep-2022
  • (2022)Unified many-worlds browsing of arbitrary physics-based animationsACM Transactions on Graphics10.1145/3528223.353008241:4(1-15)Online publication date: 22-Jul-2022
  • (2022)Learning Visual Styles from Audio-Visual AssociationsComputer Vision – ECCV 202210.1007/978-3-031-19836-6_14(235-252)Online publication date: 23-Oct-2022
  • (2020)Fizik Tabanlı Ses Sentezi Uygulamaları Üzerine Bir İncelemeHaliç Üniversitesi Fen Bilimleri Dergisi10.46373/hafebid.7826393:2(289-305)Online publication date: 30-Sep-2020
  • (2019)Animation Synthesis Triggered by Vocal MimicsProceedings of the 12th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3359566.3360067(1-5)Online publication date: 28-Oct-2019
  • (2019)Toward Generating Realistic Sounds for Soft BodiesProceedings of the 14th International Audio Mostly Conference: A Journey in Sound10.1145/3356590.3356620(199-206)Online publication date: 18-Sep-2019
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media