skip to main content
research-article

Data-Driven Particle-Based Liquid Simulation with Deep Learning Utilizing Sub-Pixel Convolution

Published: 28 April 2021 Publication History

Abstract

In recent years, the performance of neural network inference has been drastically improved. This rapid change has paved the way for research projects focusing on accelerating physics-based simulations by replacing solver with its approximation. In this paper, we propose several efficient architectures of neural networks, which can be used to exploit this idea. The purpose of our research was to specifically target a liquid simulation problem. The central challenge for us was to create an efficient solution capable of approximating Position Based Fluid [Macklin and Müller 2013] solver. It requires the network to produce an accurate output at particles located in a continuous space and be significantly faster than the GPU based simulation. We achieved this by using modern sub-pixel convolution techniques originally used for image super-resolution. In our experiments, our method runs up to 200 times faster than the reference GPU simulation.

Supplementary Material

tumanov (tumanov.zip)
Supplemental movie, appendix, image and software files for, Data-Driven Particle-Based Liquid Simulation with Deep Learning Utilizing Sub-Pixel Convolution

References

[1]
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018. Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261 [cs.LG]
[2]
Markus Becker and Matthias Teschner. 2007. Weakly Compressible SPH for Free Surface Flows. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (San Diego, California) (SCA '07). Eurographics Association, Goslar, DEU, 209--217.
[3]
Jan Bender and Dan Koschier. 2015. Divergence-Free Smoothed Particle Hydrodynamics. In Proceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA '15). Association for Computing Machinery, New York, NY, USA, 147--155.
[4]
Blender Online Community. 2018. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam. http://www.blender.org
[5]
Douglas Enright, Stephen Marschner, and Ronald Fedkiw. 2002. Animation and rendering of complex water surfaces. ACM Trans. Graph. 21, 3 (July 2002), 736--744.
[6]
Nick Foster and Ronald Fedkiw. 2001. Practical Animation of Liquids. In Proc. SIGGRAPH. 23--30.
[7]
Nick Foster and Dimitri Metaxas. 1996. Realistic animation of liquids. Graph. Models Image Process. 58, 5 (1996), 471--483.
[8]
Chuyuan Fu, Qi Guo, Theodore Gast, Chenfanfu Jiang, and Joseph Teran. 2017. A Polynomial Particle-in-Cell Method. ACM Trans. Graph. 36, 6, Article 222 (Nov. 2017), 12 pages.
[9]
K. Fukushima. 1980. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics 36 (1980), 193--202.
[10]
Robert A Gingold and Joseph J Monaghan. 1977. Smoothed particle hydrodynamics: theory and application to non-spherical stars. Monthly notices of the royal astronomical society 181, 3 (1977), 375--389.
[11]
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. 2018. A Moving Least Squares Material Point Method with Displacement Discontinuity and Two-Way Rigid Body Coupling. ACM Trans. Graph. 37, 4, Article 150 (July 2018), 14 pages.
[12]
M. Ihmsen, J. Cornelis, B. Solenthaler, C. Horvath, and M. Teschner. 2014. Implicit Incompressible SPH. IEEE Transactions on Visualization and Computer Graphics 20, 3 (2014), 426--435.
[13]
S. Ji, W. Xu, M. Yang, and K. Yu. 2013. 3D Convolutional Neural Networks for Human Action Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1 (2013), 221--231.
[14]
Chenfanfu Jiang, Craig Schroeder, Andrew Selle, Joseph Teran, and Alexey Stomakhin. 2015. The Affine Particle-in-Cell Method. ACM Trans. Graph. 34, 4, Article 51 (July 2015), 10 pages.
[15]
Asifullah Khan, Anabia Sohail, Umme Zahoora, and Aqsa Saeed Qureshi. 2020. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review (Apr 2020).
[16]
L'ubor Ladicky, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1--9.
[17]
Miles Macklin and Matthias Müller. 2013. Position based fluids. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--12.
[18]
Miles Macklin, Matthias Müller, and Richard Tonge. 2017. Nvidia FleX. https://docs.nvidia.com/gameworks/content/gameworkslibrary/physx/flex/index.html.
[19]
Matthias Müller, David Charypar, and Markus Gross. 2003. Particle-Based Fluid Simulation for Interactive Applications. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (San Diego, California) (SCA '03). Eurographics Association, Goslar, DEU, 154--159.
[20]
Simon Premžoe, Tolga Tasdizen, James Bigler, Aaron Lefohn, and Ross T. Whitaker. 2003. Particle-Based Simulation of Fluids. Computer Graphics Forum 22, 3 (2003), 401--410.
[21]
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. 2020. Learning to Simulate Complex Physics with Graph Networks. arXiv:2002.09405 [cs.LG]
[22]
Lin Shi and Yizhou Yu. 2004. Inviscid and Incompressible Fluid Simulation of Triangle Meshes. Computer Animation and Virtual Worlds 15, 3--4 (2004), 173--181.
[23]
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1874--1883.
[24]
Ilia Sivkov, Patrick Seewald, Alfio Lazzaro, and Juerg Hutter. 2019. DBCSR: A Blocked Sparse Tensor Algebra Library. arXiv:1910.13555 [cs.DC]
[25]
B. Solenthaler and R. Pajarola. 2009. Predictive-Corrective Incompressible SPH. In ACM SIGGRAPH 2009 Papers (New Orleans, Louisiana) (SIGGRAPH '09). Association for Computing Machinery, New York, NY, USA, Article 40, 6 pages.
[26]
J. Stam. 1999. Stable Fluids. In Proc. SIGGRAPH. 121--128.
[27]
Alexey Stomakhin, Craig Schroeder, Lawrence Chai, Joseph Teran, and Andrew Selle. 2013. A Material Point Method for Snow Simulation. ACM Trans. Graph. 32, 4, Article 102 (July 2013), 10 pages.
[28]
A. Stomakhin, C. Schroeder, Chenfanfu Jiang, Lawrence Chai, J. Teran, and A. Selle. 2014. Augmented MPM for phase-change and varied materials. ACM Trans. Graph. 33 (2014), 138:1--138:11.
[29]
D. Sulsky, Z. Chen, and H.L. Schreyer. 1994. A particle method for history-dependent materials. Computer Methods in Applied Mechanics and Engineering 118, 1 (1994), 179--196.
[30]
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2017. Accelerating eulerian fluid simulation with convolutional networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 3424--3433.
[31]
Benjamin Ummenhofer, Lukas Prantl, Nils Thürey, and Vladlen Koltun. 2020. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations.
[32]
Marcel Weiler, Dan Koschier, and Jan Bender. 2016. Projective Fluids. In Proceedings of the 9th International Conference on Motion in Games (Burlingame, California) (MIG '16). Association for Computing Machinery, New York, NY, USA, 79--84.
[33]
Steffen Wiewel, Moritz Becher, and Nils Thuerey. 2019. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 71--82.
[34]
Matthew D Zeiler, Dilip Krishnan, Graham W Taylor, and Rob Fergus. 2010. Deconvolutional networks. In 2010 IEEE Computer Society Conference on computer vision and pattern recognition. IEEE, 2528--2535.
[35]
Yongning Zhu and Robert Bridson. 2005. Animating Sand as a Fluid. In the Proceedings of ACM SIGGRAPH 2005. 965--972.

Cited By

View all
  • (2022)Evaluating the Stability of Numerical Schemes for Fluid Solvers in Game TechnologyInternational Journal of Computer Games Technology10.1155/2022/41383152022Online publication date: 1-Jan-2022

Index Terms

  1. Data-Driven Particle-Based Liquid Simulation with Deep Learning Utilizing Sub-Pixel Convolution

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
      Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 4, Issue 1
      April 2021
      274 pages
      EISSN:2577-6193
      DOI:10.1145/3463840
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 28 April 2021
      Published in PACMCGIT Volume 4, Issue 1

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. data-driven simulation
      2. neural networks
      3. real-time fluid simulation
      4. simulation super resolution

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)68
      • Downloads (Last 6 weeks)8
      Reflects downloads up to 06 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Evaluating the Stability of Numerical Schemes for Fluid Solvers in Game TechnologyInternational Journal of Computer Games Technology10.1155/2022/41383152022Online publication date: 1-Jan-2022

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media