skip to main content
research-article

Toonsynth: example-based synthesis of hand-colored cartoon animations

Published: 30 July 2018 Publication History

Abstract

We present a new example-based approach for synthesizing hand-colored cartoon animations. Our method produces results that preserve the specific visual appearance and stylized motion of manually authored animations without requiring artists to draw every frame from scratch. In our framework, the artist first stylizes a limited set of known source skeletal animations from which we extract a style-aware puppet that encodes the appearance and motion characteristics of the artwork. Given a new target skeletal motion, our method automatically transfers the style from the source examples to create a hand-colored target animation. Compared to previous work, our technique is the first to preserve both the detailed visual appearance and stylized motion of the original hand-drawn content. Our approach has numerous practical applications including traditional animation production and content creation for games.

Supplementary Material

ZIP File (167-287.zip)
Supplemental files.
MP4 File (167-287.mp4)
MP4 File (a167-dvoroznak.mp4)

References

[1]
Rahul Arora, Ishan Darolia, Vinay Namboodiri, Karan Singh, and Adrien Bousseau. 2017. SketchSoup: Exploratory Ideation Using Design Sketches. Computer Graphics Forum 36, 8 (2017), 302--312.
[2]
Yunfei Bai, Danny M Kaufman, Karen Liu, and Jovan Popović. 2016. Artist-directed dynamics for 2D animation. ACM Transactions on Graphics 35, 4 (2016), 145.
[3]
William Baxter and Ken-ichi Anjyo. 2006. Latent Doodle Space. Computer Graphics Forum 25, 3 (2006), 477--485.
[4]
William Baxter, Pascal Barla, and Ken Anjyo. 2009. N-way morphing for 2D animation. Journal of Visualization and Computer Animation 20, 2--3 (2009), 79--87.
[5]
Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden. 2013. Stylizing animation by example. ACM Transactions on Graphics 32, 4 (2013), 119.
[6]
Mikhail Bessmeltsev, Nicholas Vining, and Alla Sheffer. 2016. Gesture3D: posing 3D characters via gesture drawings. ACM Transactions on Graphics 35, 6 (2016), 165.
[7]
Christoph Bregler, Lorie Loeb, Erika Chuang, and Hrishi Deshpande. 2002. Turning to the Masters: Motion Capturing Cartoons. ACM Transactions on Graphics 21, 3 (2002), 399--407.
[8]
Ian Buck, Adam Finkelstein, Charles Jacobs, Allison Klein, David Salesin, Joshua Seims, Richard Szeliski, and Kentaro Toyama. 2000. Performance-Driven Hand-Drawn Animation. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 101--108.
[9]
Nestor Burtnyk and Marceli Wein. 1976. Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation. Commun. ACM 19, 10 (1976), 564--569.
[10]
Edwin Catmull. 1978. The Problems of Computer-Assisted Animation. 12, 3 (1978), 348--353.
[11]
James Davis, Maneesh Agrawala, Erika Chuang, Zoran Popovic, and David Salesin. 2003. A sketching interface for articulated figure animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 320--328.
[12]
Marek Dvorožňák, Pierre Bénard, Pascal Barla, Oliver Wang, and Daniel Sýkora. 2017. Example-Based Expressive Animation of 2D Rigid Bodies. ACM Transactions on Graphics 36, 4, Article 127 (2017).
[13]
Elmar Eisemann and Frédo Durand. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (2004), 673--678.
[14]
Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Eli Shechtman, Paul Asente, Jingwan Lu, and Daniel Sýkora. 2016. Stylit: Illumination-guided Example-based Stylization of 3D Renderings. ACM Transactions on Graphics 35, 4 (2016), 92.
[15]
Jakub Fišer, Ondřej Jamriška, David Simons, Eli Shechtman, Jingwan Lu, Paul Asente, Michal Lukáč, and Daniel Sýkora. 2017. Example-Based Synthesis of Stylized Facial Animations. ACM Transactions on Graphics 36, 4, Article 155 (2017).
[16]
Jakub Fišer, Michal Lukáč, Ondřej Jamriška, Martin Čadík, Yotam Gingold, Paul Asente, and Daniel Sýkora. 2014. Color Me Noisy: Example-Based Rendering of Hand-Colored Animations with Temporal Noise Control. Computer Graphics Forum 33, 4 (2014), 1--10.
[17]
Ben Glocker, Nikos Komodakis, Georgios Tziritas, Nassir Navab, and Nikos Paragios. 2008. Dense Image Registration Through MRFs And Efficient Linear Programming. Medical Image Analysis 12, 6 (2008), 731--741.
[18]
William van Haevre, Fabian di Fiore, and Frank van Reeth. 2005. Uniting Cartoon Textures with Computer Assisted Animation. In Proceedings of International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia. 245--253.
[19]
Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. 2001. Image analogies. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 327--340.
[20]
Alexander Hornung, Ellen Dekkers, and Leif Kobbelt. 2007. Character Animation from 2D Pictures and 3D Motion Data. ACM Transactions on Graphics 26, 1 (2007).
[21]
Eakta Jain, Yaser Sheikh, and Jessica Hodgins. 2009. Leveraging the talent of hand animators to create three-dimensional animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 93--102.
[22]
Ben Jones Jovan Popovic, James McCann, Wilmot Li, and Adam Bargteil. 2015. Dynamic sprites: Artistic authoring of interactive animations. Journal of Visualization and Computer Animation 26, 2 (2015), 97--108.
[23]
Christina de Juan and Bobby Bodenheimer. 2004. Cartoon Textures. In Proceedings of Eurographics Symposium on Computer Animation. 267--276.
[24]
Christina de Juan and Bobby Bodenheimer. 2006. Re-using traditional animation: Methods for semi-automatic segmentation and inbetweening. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 223--232.
[25]
Rubaiat Habib Kazi, Tovi Grossman, Nobuyuki Umetani, and George Fitzmaurice. 2016. Motion Amplifiers: Sketching Dynamic Illustrations Using the Principles of 2D Animation. In Proceedings of ACM Conference on Human Factors in Computing Systems. 4599--4609.
[26]
Alexander Kort. 2002. Computer Aided Inbetweening. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 125--132.
[27]
Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3 (2002), 473--482.
[28]
Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics 24, 3 (2005), 795--802.
[29]
John Lasseter. 1987. Principles of Traditional Animation Applied to 3D Computer Animation. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. 35--44.
[30]
Seungyong Lee, George Wolberg, and Sung Yong Shin. 1998. Polymorph: Morphing Among Multiple Images. IEEE Computer Graphics and Applications 18, 1 (1998), 58--71.
[31]
Sun-Young Lee, Jong-Chul Yoon, Ji-Yong Kwon, and In-Kwon Lee. 2012. CartoonModes: Cartoon Stylization of Video Objects Through Modal Analysis. Graphical Models 74, 2 (2012), 51--60.
[32]
Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics 36, 4 (2017), 44:1--44:14.
[33]
Johannes Schmid, Robert Sumner, Huw Bowles, and Markus Gross. 2010. Programmable Motion Effects. ACM Transactions on Graphics 29, 4 (2010), 57.
[34]
Daniel Sýkora, Mirela Ben-Chen, Martin Čadík, Brian Whited, and Maryann Simmons. 2011. TexToons: Practical Texture Mapping for Hand-drawn Cartoon Animations. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 75--83.
[35]
Daniel Sýkora, Jan Buriánek, and Jiří Žára. 2005. Sketching Cartoons by Example. In Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling. 27--34.
[36]
Daniel Sýkora, John Dingliana, and Steven Collins. 2009. As-Rigid-As-Possible Image Registration for Hand-Drawn Cartoon Animations. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 25--33.
[37]
Daniel Sýkora, Ladislav Kavan, Martin Čadík, Ondřej Jamriška, Alec Jacobson, Brian Whited, Maryann Simmons, and Olga Sorkine-Hornung. 2014. Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters. ACM Transactions on Graphics 33, 2 (2014), 16.
[38]
Daniel Sýkora, David Sedlacek, Sun Jinchao, John Dingliana, and Steven Collins. 2010. Adding Depth to Cartoons Using Sparse Depth (In)equalities. Computer Graphics Forum 29, 2 (2010), 615--623.
[39]
Cedric Vanaken, Chris Hermans, Tom Mertens, Fabian Di Fiore, Philippe Bekaert, and Frank Van Reeth. 2008. Strike a Pose: Image-Based Pose Synthesis. In Proceedings of the Conference on Vision, Modeling and Visualization. 131--138.
[40]
Jue Wang, Steven Drucker, Maneesh Agrawala, and Michael Cohen. 2006. The Cartoon Animation Filter. ACM Transactions on Graphics 25, 3 (2006), 1169--1173.
[41]
Xun Wang, Wenwu Yang, Haoyu Peng, and Guozheng Wang. 2013. Shape-aware skeletal deformation for 2D characters. The Visual Computer 29, 6--8 (2013), 545--553.
[42]
Yonatan Wexler, Eli Shechtman, and Michal Irani. 2007. Space-Time Completion of Video. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 3 (2007), 463--476.
[43]
Brian Whited, Gioacchino Noris, Maryann Simmons, Robert Sumner, Markus Gross, and Jarek Rossignac. 2010. BetweenIT: An Interactive Tool for Tight Inbetweening. Computer Graphics Forum 29, 2 (2010), 605--614.
[44]
Nora Willett, Wilmot Li, Jovan Popovic, Floraine Berthouzoz, and Adam Finkelstein. 2017. Secondary Motion for Performed 2D Animation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). ACM, New York, NY, USA, 97--108.
[45]
Wenwu Yang. 2017. Context-Aware Computer Aided Inbetweening. IEEE Transactions on Visualizaton and Computer Graphics (2017).
[46]
Chih-Kuo Yeh, Shi-Yang Huang, Pradeep Kumar Jayaraman, Chi-Wing Fu, and Tong-Yee Lee. 2017. Interactive High-Relief Reconstruction for Organic and Double-Sided Objects from a Photo. IEEE Transactions on Visualization and Computer Graphics 23, 7 (2017), 1796--1808.
[47]
Lei Zhang, Hua Huang, and Hongbo Fu. 2012. EXCOL: An EXtract-and-COmplete Layering Approach to Cartoon Animation Reusing. IEEE Transactions on Visualization and Computer Graphics 18, 7 (2012), 1156--1169.
[48]
Yufeng Zhu, Jovan Popović, Robert Bridson, and Danny Kaufman. 2017. Planar Interpolation with Extreme Deformation, Topology Change and Dynamics. ACM Transactions on Graphics 36, 6 (2017), 213.

Cited By

View all
  • (2024)Skeleton-Driven Inbetweening of Bitmap Character DrawingsACM Transactions on Graphics10.1145/368795543:6(1-19)Online publication date: 19-Dec-2024
  • (2024)Joint Stroke Tracing and Correspondence for 2D AnimationACM Transactions on Graphics10.1145/364989043:3(1-17)Online publication date: 9-Apr-2024
  • (2024)Breathing Life Into Sketches Using Text-to-Video Priors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00414(4325-4336)Online publication date: 16-Jun-2024
  • Show More Cited By

Index Terms

  1. Toonsynth: example-based synthesis of hand-colored cartoon animations

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 37, Issue 4
      August 2018
      1670 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3197517
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 July 2018
      Published in TOG Volume 37, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. skeletal animation
      2. style transfer

      Qualifiers

      • Research-article

      Funding Sources

      • Technology Agency of the Czech Republic
      • Fulbright Scholarship
      • Grant Agency of the Czech Technical University in Prague

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)37
      • Downloads (Last 6 weeks)8
      Reflects downloads up to 16 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Skeleton-Driven Inbetweening of Bitmap Character DrawingsACM Transactions on Graphics10.1145/368795543:6(1-19)Online publication date: 19-Dec-2024
      • (2024)Joint Stroke Tracing and Correspondence for 2D AnimationACM Transactions on Graphics10.1145/364989043:3(1-17)Online publication date: 9-Apr-2024
      • (2024)Breathing Life Into Sketches Using Text-to-Video Priors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00414(4325-4336)Online publication date: 16-Jun-2024
      • (2023)A Method for Animating Children’s Drawings of the Human FigureACM Transactions on Graphics10.1145/359278842:3(1-15)Online publication date: 5-Jun-2023
      • (2023)Efficient Interpolation of Rough Line DrawingsComputer Graphics Forum10.1111/cgf.1494642:7Online publication date: 5-Nov-2023
      • (2023)Non‐linear Rough 2D Animation using Transient EmbeddingsComputer Graphics Forum10.1111/cgf.1477142:2(411-425)Online publication date: 23-May-2023
      • (2022)Sprite-from-SpriteACM Transactions on Graphics10.1145/3550454.355543941:6(1-12)Online publication date: 30-Nov-2022
      • (2022)Deep Sketch-Guided Cartoon Video InbetweeningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.304941928:8(2938-2952)Online publication date: 1-Aug-2022
      • (2021)STALP: Style Transfer with Auxiliary Limited PairingComputer Graphics Forum10.1111/cgf.14265540:2(563-573)Online publication date: 4-Jun-2021
      • (2021)Video Snapshot: Single Image Motion Expansion via Invertible Motion EmbeddingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2020.300164443:12(4491-4504)Online publication date: 1-Dec-2021
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media