skip to main content
research-article

Selectively de-animating video

Published: 01 July 2012 Publication History

Abstract

We present a semi-automated technique for selectively deanimating video to remove the large-scale motions of one or more objects so that other motions are easier to see. The user draws strokes to indicate the regions of the video that should be immobilized, and our algorithm warps the video to remove the large-scale motion of these regions while leaving finer-scale, relative motions intact. However, such warps may introduce unnatural motions in previously motionless areas, such as background regions. We therefore use a graph-cut-based optimization to composite the warped video regions with still frames from the input video; we also optionally loop the output in a seamless manner. Our technique enables a number of applications such as clearer motion visualization, simpler creation of artistic cinemagraphs (photos that include looping motions in some regions), and new ways to edit appearance and complicated motion paths in video by manipulating a de-animated representation. We demonstrate the success of our technique with a number of motion visualizations, cinemagraphs and video editing examples created from a variety of short input videos, as well as visual and numerical comparison to previous techniques.

Supplementary Material

JPG File (tp167_12.jpg)
ZIP File (a66-bai.zip)
Supplemental material.
MP4 File (tp167_12.mp4)

References

[1]
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3 (Aug.), 294--302.
[2]
Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D. H., and Szeliski, R. 2005. Panoramic video textures. ACM Transactions on Graphics 24, 3 (Aug.), 821--827.
[3]
Assa, J., Caspi, Y., and Cohen-Or, D. 2005. Action synopsis: pose selection and illustration. ACM Transactions on Graphics 24, 3 (Aug.), 667--676.
[4]
Bai, X., Wang, J., Simons, D., and Sapiro, G. 2009. Video snapcut: Robust video object cutout using localized classifiers. ACM Transactions on Graphics 28, 3 (July), 70:1--70:11.
[5]
Barnes, C., Goldman, D. B., Shechtman, E., and Finkelstein, A. 2010. Video tapestries with continuous temporal zoom. ACM Transactions on Graphics 29, 4 (July), 89:1--89:9.
[6]
Bennett, E. P., and McMillan, L. 2007. Computational time-lapse video. ACM Transactions on Graphics 26, 3 (July), 102:1--102:6.
[7]
Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 11 (Nov.), 1222--1239.
[8]
Caspi, Y., Axelrod, A., Matsushita, Y., and Gamliel, A. 2006. Dynamic stills and clip trailers. The Visual Computer 22, 9--10, 642--652.
[9]
Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., and Szeliski, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3 (July), 243--248.
[10]
Chuang, Y.-Y., Goldman, D. B., Zheng, K. C., Curless, B., Salesin, D. H., and Szeliski, R. 2005. Animating pictures with stochastic motion textures. ACM Transactions on Graphics 24, 3 (Aug.), 853--860.
[11]
Cohen, M. F., and Szeliski, R. 2006. The moment camera. Computer 39, 8, 40--45.
[12]
Correa, C. D., and Ma, K.-L. 2010. Dynamic video narratives. ACM Transactions on Graphics 29, 4 (July), 88:1--88:9.
[13]
Goldman, D. B., Curless, B., Salesin, D., and Seitz, S. M. 2006. Schematic storyboarding for video visualization and editing. ACM Transactions on Graphics 25, 3 (July), 862--871.
[14]
Kim, B., and Essa, I. 2005. Video-based nonphotorealistic and expressive illustration of motion. In Computer Graphics International 2005.
[15]
Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics 22, 3 (July), 277--286.
[16]
Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Transactions on Graphics 24, 3 (Aug.), 519--526.
[17]
Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Transactions on Graphics 28, 3 (July), 44:1--44:9.
[18]
Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Transactions on Graphics 30, 1 (Jan.), 4:1--4:10.
[19]
Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. International Joint Conference on Artificial Intelligence.
[20]
Pritch, Y., Rav-Acha, A., and Peleg, S. 2008. Nonchronological video synopsis and indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 11 (Nov.), 1971--1984.
[21]
Rav-Acha, A., Kohli, P., Rother, C., and Fitzgibbon, A. 2008. Unwrap mosaics: A new representation for video editing. ACM Transactions on Graphics 27, 3 (Aug.), 17:1--17:11.
[22]
Rubinstein, M., Liu, C., Sand, P., Durand, F., and Freeman, W. T. 2011. Motion denoising with application to time-lapse photography. IEEE Computer Vision and Pattern Recognition (CVPR) (June), 313--320.
[23]
Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 489--498.
[24]
Shi, J., and Tomasi, C. 1994. Good features to track. In Computer Vision and Pattern Recognition, 593--600.
[25]
Sutton, G. P., and Burrows, M. 2011. Biomechanics of jumping in the flea. J Exp Biol 214, 5 (Mar.), 836--847.
[26]
Tompkin, J., Pece, F., Subr, K., and Kautz, J. 2011. Towards moment imagery: Automatic cinemagraphs. Visual Media Production, Conference for 0, 87--93.
[27]
Truong, B. T., and Venkatesh, S. 2007. Video abstraction: A systematic review and classification. ACM Trans. Multimedia Comput. Commun. Appl. 3, 1 (Feb.).

Cited By

View all
  • (2023)Lagrangian motion magnification with double sparse optical flow decompositionFrontiers in Applied Mathematics and Statistics10.3389/fams.2023.11644919Online publication date: 14-Sep-2023
  • (2023)Text-Guided Synthesis of Eulerian CinemagraphsACM Transactions on Graphics10.1145/361832642:6(1-13)Online publication date: 5-Dec-2023
  • (2023)Hierarchical Planning and Control for Box Loco-ManipulationProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069316:3(1-18)Online publication date: 24-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 31, Issue 4
July 2012
935 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/2185520
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012
Published in TOG Volume 31, Issue 4

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)1
Reflects downloads up to 23 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Lagrangian motion magnification with double sparse optical flow decompositionFrontiers in Applied Mathematics and Statistics10.3389/fams.2023.11644919Online publication date: 14-Sep-2023
  • (2023)Text-Guided Synthesis of Eulerian CinemagraphsACM Transactions on Graphics10.1145/361832642:6(1-13)Online publication date: 5-Dec-2023
  • (2023)Hierarchical Planning and Control for Box Loco-ManipulationProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069316:3(1-18)Online publication date: 24-Aug-2023
  • (2023)Generating Activity Snippets by Learning Human-Scene InteractionsACM Transactions on Graphics10.1145/359209642:4(1-15)Online publication date: 26-Jul-2023
  • (2023)Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00052(459-468)Online publication date: Jun-2023
  • (2022)Interactive augmented reality storytelling guided by scene semanticsACM Transactions on Graphics10.1145/3528223.353006141:4(1-15)Online publication date: 22-Jul-2022
  • (2022)Visualizing and quantifying small and nonstationary structural motions in video measurementComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1289438:2(135-159)Online publication date: 29-Jul-2022
  • (2021)Joint computational design of workspaces and workplansACM Transactions on Graphics10.1145/3478513.348050040:6(1-16)Online publication date: 10-Dec-2021
  • (2021)Endless loopsACM Transactions on Graphics10.1145/3450626.345993540:4(1-12)Online publication date: 19-Jul-2021
  • (2021)Video‐Based Rendering of Dynamic Stationary Environments from Unsynchronized InputsComputer Graphics Forum10.1111/cgf.1434240:4(73-86)Online publication date: 15-Jul-2021
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media