Skip to main content
Springer Nature Link
Log in

Lossy data compression reduces communication time in hybrid time-parallel integrators

  • Special Issue Parallel-in-Time Methods
  • Published:
Computing and Visualization in Science

Abstract

Parallel-in-time methods for solving initial value problems are a means to increase the parallelism of numerical simulations. Hybrid parareal schemes interleaving the parallel-in-time iteration with an iterative solution of the individual time steps are among the most efficient methods for general nonlinear problems. Despite the hiding of communication time behind computation, communication has in certain situations a significant impact on the total runtime. Here we present strict, yet not sharp, error bounds for hybrid parareal methods with inexact communication due to lossy data compression, and derive theoretical estimates of the impact of compression on parallel efficiency of the algorithms. These and some computational experiments suggest that compression is a viable method to make hybrid parareal schemes robust with respect to low bandwidth setups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alhubail, M., Wang, Q., Williams, J.: The swept rule for breaking the latency barrier in time advancing two-dimensional PDEs. Preprint (2016). arXiv:1602.07558

  2. Alhubail, M., Wang, Q.: The swept rule for breaking the latency barrier in time advancing PDEs. J. Comput. Phys. 307, 110–121 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Aubanel, E.: Scheduling of tasks in the parareal algorithm. Parallel Comput. 37, 172–182 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bal, G.: On the convergence and the stability of the parareal algorithm to solve partial differential equations. In: Kornhuber, R., Hoppe, R., Périaux, J., Pironneau, O., Widlund, O., Xu, J. (eds.) Proceedings of DD15, volume 40 of Lecture Notes in Computational Science and Engineering, pp. 425–432. Springer (2004)

  5. Barker, A.T.: A minimal communication approach to parallel time integration. Int. J. Comput. Math. 91(3), 601–615 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bolten, M., Moser, D., Speck, R.: Asymptotic convergence of the parallel full approximation scheme in space and time for linear problems. Preprint (2017). arXiv:1703.07120

  7. Christlieb, A.J., Macdonald, C.B., Ong, B.W.: Parallel high-order integrators. SIAM J. Sci. Comput. 32(2), 818–835 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  8. Duarte, M., Massot, M., Descombes, S.: Parareal operator splitting techniques for multi-scale reaction waves: numerical analysis and strategies. M2AN 45(5), 825–852 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dutt, A., Greengard, L., Rokhlin, V.: Spectral deferred correction methods for ordinary differential equations. BIT 40(2), 241–266 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  10. Emmett, M., Minion, M.L.: Toward an efficient parallel in time method for partial differential equations. Commun. Appl. Math. Comput. Sci. 7(1), 105–132 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Filgueira, R., Singh, D.E., Carretero, J., Calderón, A., García, F.: Adaptive-compi: enhancing MPI-based applications’ performance and scalability by using adaptive compression. Int. J. High Perform. Comput. Appl. 25(1), 93–114 (2011)

    Article  Google Scholar 

  12. Gander, M.: 50 years of time parallel time integration. In: Carraro, T., Geiger, M., Körkel, S., Rannacher, R. (eds.) Multiple Shooting and Time Domain Decomposition Methods, volume 9 of Contributions in Mathematical and Computational Sciences, pp. 69–113. Springer (2015)

  13. Gander, M.J., Hairer, E.: Nonlinear convergence analysis for the parareal algorithm. In: Langer, U., Discacciati, M., Keyes, D.E., Widlund, O.B., Zulehner, W. (eds.) Domain Decomposition Methods in Science and Engineering XVII, pp. 45–56. Springer, Berlin (2008)

    Chapter  Google Scholar 

  14. Gander, M.J., Neumüller, M.: Analysis of a new space-time parallel multigrid algorithm for parabolic problems. SIAM J. Sci. Comput. 38(4), A2173–A2208 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gander, M.J., Vandewalle, S.: Analysis of the parareal time-parallel time-integration method. SIAM J. Sci. Comput. 29(2), 556–578 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Götschel, S., Weiser, M.: Lossy compression for PDE-constrained optimization: adaptive error control. Comput. Optim. Appl. 62, 131–155 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  17. Götschel, S., Weiser, M., Schiela, A.: Solving optimal control problems with the Kaskade 7 finite element toolbox. In: Dedner, A., Flemisch, B., Klöfkorn, R. (eds.) Advances in DUNE, pp. 101–112. Springer, Berlin (2012)

    Chapter  Google Scholar 

  18. Guibert, D., Tromeur-Dervout, D.: Parallel deferred correction method for CFD problems. In: Kwon, J.-H., Periaux, J., Fox, P., Satofuka, N., Ecer, A. (eds.) Parallel Computational Fluid Dynamics 2006: Parallel Computing and Its Applications, pp. 131–136 (2007)

  19. Ke, J., Burtscher, M., Speight, E.: Runtime compression of MPI messages to improve the performance and scalability of parallel applications. In: Supercomputing, 2004. Proceedings of the ACM/IEEE SC2004 Conference, p. 59 (2004)

  20. Keckler, S.W., Dally, W.J., Khailany, B., Garland, M., Glasco, D.: Gpus and the future of parallel computing. IEEE Micro 31(5), 7–17 (2011)

    Article  Google Scholar 

  21. Klatt, T., Emmett, M., Ruprecht, D., Speck, R., Terzi, S.: PFASST++. http://www.parallelintime.org/codes/pfasst.html (2015). Retrieved: 04 May 2017

  22. Leyffer, S., Wild, S.M., Fagan, M., Snir, M., Palem, K., Yoshii, K., Finkel, H.: Doing Moore with less—Leapfrogging Moore’s law with inexactness for supercomputing. CoRR (2016). arXiv:1610.02606

  23. Lions, J.-L., Maday, Y., Turinici, G.: A parareal in time discretization of pdes. C. R. Acad. Sci. Paris Ser. I 332, 661–668 (2001)

    Article  MATH  Google Scholar 

  24. Liu, J., Wang, Y., Li, R.: A hybrid algorithm based on optimal quadratic spline collocation and parareal deferred correction for parabolic PDEs. Math. Probl. Eng. Article ID 6943079 (2016)

  25. McDonald, E., Wathen, A.J.: A simple proposal for parallel computation over time of an evolutionary process with implicit time stepping. In: Proceedings of the ENUMATH 2015, Lecture Notes in Computational Science and Engineering. Springer (2016)

  26. Minion, M.L.: A hybrid parareal spectral deferred corrections method. Commun. Appl. Math. Comput. Sci. 5(2), 265–301 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  27. Nielsen, A.S., Brunner, G., Hesthaven, J.S.: Communication-aware adaptive parareal with application to a nonlinear hyperbolic system of partial differential equations. EPFL-ARTICLE 228189, EPFL (2017)

  28. Núñez, A., Filgueira, R., Merayo, M.G.: SANComSim: a scalable, adaptive and non-intrusive framework to optimize performance in computational science applications. Proc. Comput. Sci. 18, 230–239 (2013)

    Article  Google Scholar 

  29. Palmer, T.: Build imprecise supercomputers. Nature 526, 32–33 (2015)

    Article  Google Scholar 

  30. Ruprecht, D.: Wave propagation characteristics of Parareal. Comput. Vis. Sci. (2017)

  31. Saravanan, K.P., Carpenter, P.M., Ramirez, A.: A performance perspective on energy efficient HPC links. In: ICS ’14 Proceedings of the 28th ACM International Conference on Supercomputing, pp. 313–322 (2014)

  32. Srinivasan, A., Chandra, N.: Latency tolerance through parallelization of time in scientific applications. Parallel Comput. 31, 777–796 (2005)

    Article  Google Scholar 

  33. Toselli, A., Widlund, O.B.: Domain Decomposition Methods-Algorithms and Theory, volume 34 of Computational Mathematics. Springer, Berlin (2005)

    Book  MATH  Google Scholar 

  34. Weiser, M.: Faster SDC convergence on non-equidistant grids by DIRK sweeps. BIT Numer. Math. 55(4), 1219–1241 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  35. Weiser, M., Götschel, S.: State trajectory compression for optimal control with parabolic PDEs. SIAM J. Sci. Comput. 34(1), A161–A184 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wu, S.-L., Zhou, T.: Convergence analysis for three parareal solvers. SIAM J. Sci. Comput. 37(2), A970–A992 (2015)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Partial funding by BMBF Project SOAK and DFG Project WE 2937/6-1 is gratefully acknowledged. The authors would also like to thank Th. Steinke, F. Wende, A. Kammeyer and the North-German Supercomputing Alliance (HLRN) for supporting the numerical experiments.

Author information

Authors and Affiliations

  1. Zuse Institute Berlin, Takustr. 7, 14195, Berlin, Germany

    Lisa Fischer, Sebastian Götschel & Martin Weiser

Authors
  1. Lisa Fischer
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sebastian Götschel
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Martin Weiser
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Martin Weiser.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fischer, L., Götschel, S. & Weiser, M. Lossy data compression reduces communication time in hybrid time-parallel integrators. Comput. Visual Sci. 19, 19–30 (2018). https://doi.org/10.1007/s00791-018-0293-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00791-018-0293-2

Keywords