Skip to main content
SpringerLink
Log in

Parallel Domain Decomposition Methods with Mixed Order Discretization for Fully Implicit Solution of Tracer Transport Problems on the Cubed-Sphere

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, a fully implicit finite volume Eulerian scheme and a corresponding scalable parallel solver are developed for some tracer transport problems on the cubed-sphere. To efficiently solve the large sparse linear system at each time step on parallel computers, we introduce a Schwarz preconditioned Krylov subspace method using two discretizations. More precisely speaking, the higher order method is used for the residual calculation and the lower order method is used for the construction of the preconditioner. The matrices from the two discretizations have similar sparsity pattern and eigenvalue distributions, but the matrix from the lower order method is a lot sparser, as a result, excellent scalability results (in total computing time and the number of iterations) are obtained. Even though Schwarz preconditioner is originally designed for elliptic problems, our experiments indicate clearly that the method scales well for this class of purely hyperbolic problems. In addition, we show numerically that the proposed method is highly scalable in terms of both strong and weak scalabilities on a supercomputer with thousands of processors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Balay, S., Buschelman, K., Gropp, W.D., Kaushik, D., Knepley, M., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc Users Manual. Argonne National Laboratory (2012)

  2. Brown, P.N., Shumaker, D.E., Woodward, C.S.: Fully implicit solution of large-scale non-equilibrium radiation diffusion with high order time integration. J. Comput. Phys. 204, 760–783 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Cai, X.-C., Gropp, W.D., Keyes, D.E., Melvin, R.G., Young, D.P.: Parallel Newton-Krylov-Schwarz algorithms for the transonic full potential equation. SIAM J. Sci. Comput. 19, 246–265 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cai, X.-C., Sarkis, M.: A restricted additive Schwarz preconditioner for general sparse linear systems. SIAM J. Sci. Comput. 21, 792–797 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, C., Xiao, F.: Shallow water model on cubed-sphere by multi-moment finite volume method. J. Comput. Phys. 227, 5019–5044 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Evans, K.J., Knoll, D.A.: Temporal accuracy of phase change convection simulations using the JFNK-SIMPLE algorithm. Int. J. Num. Meth. Fluids. 55, 637–655 (2007)

    Article  MATH  Google Scholar 

  7. Erath, C., Lauritzen, P.H., Garcia, J.H., Tufo, H.M.: Integrating a scalable and efficient semi-Lagrangian multi-tracer transport scheme in HOMME. Proc. Comput. Sci. 9, 994–1003 (2012)

    Article  Google Scholar 

  8. Harris, L.M., Lauritzen, P.H., Mittal, R.: A flux-form version of the conservative semi-Lagrangian multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys. 230, 1215–1237 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  9. Jacobson, M.Z.: Fundamentals of Atmospheric Modeling. Cambridge University Press, New York (1999)

    Google Scholar 

  10. Knoll, D.A., Chacon, L., Margolin, L.G., Mousseau, V.A.: On balanced approximations for time integration of multiple time scale systems. J. Comput. Phys. 185, 583–611 (2003)

    Article  MATH  Google Scholar 

  11. Lauritzen, P.H., Nair, R.D., Ullrich, P.A.: A conservative semi-Lagrangian multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys. 229, 1401–1424 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Lauritzen, P.H., Skamarock, W.C.: Test-case suite for 2D passive tracer transport: a proposal for the NCAR transport workshop. March (2011)

  13. Lauritzen, P.H., Jablonowski, C., Taylor, M., Nair, R.: Numerical Techniques for Global Atmospheric Models. Lecture Notes in Computational Science and Engineering. Springer, Berlin (2011)

    Book  Google Scholar 

  14. Lauritzen, P.H., Ullrich, P.A., Nair, R.D.: Atmospheric transport schemes: desirable properties and a semi-Lagrangian view on finite-volume discretizations. In: Lecture Notes in Computational Science and Engineering (Tutorials), vol. 80, Springer, (2011)

  15. Lauritzen, P.H., Skamarock, W.C., Prather, M.J., Taylor, M.A.: A standard test case suite for two-dimensional linear transport on the sphere. Geosci. Model Dev. Discuss. 5, 189–228 (2012)

    Article  Google Scholar 

  16. Lauritzen, P.H., Thuburn, J.: Evaluating advection/transport schemes using interrelated tracers, scatter plots and numerical mixing diagnostics. Q. J. Roy. Meteor. Soc. 138, 906–918 (2012)

    Article  Google Scholar 

  17. Nair, R.D., Thomas, S.J., Loft, R.D.: A discontinuous Galerkin global shallow water model. Mon. Weather Rev. 133, 876–888 (2005)

    Article  Google Scholar 

  18. Nair, R.D., Lauritzen, P.H.: A class of deformational-flow test cases for linear transport problems on the sphere. J. Comput. Phys. 229, 8868–8887 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Putman, W.M., Lin, S.-J.: Finite-volume transport on various cubed-sphere grids. J. Comput. Phys. 227, 55–78 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Rancic, M.R., Purser, J., Mesinger, F.: A global-shallow water model using an expanded spherical cube: Gnomonic versus conformal coordinates. Q. J. Roy. Meteor. Soc. 122, 959–982 (1996)

    Article  Google Scholar 

  21. Ronchi, C., Iacono, R., Paolucci, P.: The cubed sphere: a new method for the solution of partial differential equations in spherical geometry. J. Comput. Phys. 124, 93–114 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  22. Sadourny, R., Arakawa, A., Mintz, Y.: Integration of the nondivergent barotropic vorticity equation with an icosahedralhexagonal grid for the sphere. Mon. Weather Rev. 96, 351–356 (1968)

    Article  Google Scholar 

  23. Sadourny, R.: Conservative finite-difference approximations of the primitive equations on quasi-uniform spherical grids. Mon. Weather Rev. 100, 211–224 (1972)

    Article  Google Scholar 

  24. Saad, Y.: Iterative Methods for Sparse Linear Systems. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  25. Shadid, J.N., Tuminaro, R.S., Devine, K.D., Hennigan, G.L., Lin, P.T.: Performance of fully coupled domain decomposition preconditioners for finite element transport/reaction simulations. J. Comput. Phys. 205, 24–47 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  26. Smith, B., Bjørstad, P., Gropp, W.: Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press, Cambridge (1996)

    MATH  Google Scholar 

  27. Toselli, A., Widlund, O.: Domain Decomposition Methods-Algorithms and Theory. Springer, Berlin (2005)

    MATH  Google Scholar 

  28. Van Albada, G.D., van Leer, B., Roberts, W.W.: A comparative study of computational methods in cosmic gas dynamics. Astron. Astrophys. 108, 95–103 (1982)

    Google Scholar 

  29. White III, J.B., Dongarra, J.J.: High-performance high-resolution semi-Lagrangian tracer transport on a sphere. J. Comput. Phys. 230, 6778–6799 (2011)

    Article  MATH  Google Scholar 

  30. Williamson, D.L., Drake, J.B., Hack, J.J., Jakob, R., Swarztrauber, P.N.: A standard test set for numerical approximations to the shallow water equations in spherical geometry. J. Comput. Phys. 102, 211–224 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  31. Wu, Y., Cai, X.-C., Keyes, D.E.: Additive Schwarz methods for hyperbolic equations. In: Mandel, J., Farhat, C., Cai, X.-C. (eds.) Proceedings of the 10th International Conference on Domain Decomposition Methods, AMS, pp. 513–521 (1998)

  32. Yang, C., Cao, J., Cai, X.-C.: A fully implicit domain decomposition algorithm for shallow water equations on the cubed-sphere. SIAM J. Sci. Comput. 32, 418–438 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  33. Yang, C., Cai, X.-C.: Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on cubed-sphere. J. Comput. Phys. 230, 2523–2539 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. Yang, C., Cai, X.-C.: A scalable fully implicit compressible Euler solver for mesoscale nonhydrostatic simulation of atmospheric flows. SIAM J. Sci. Comput. To appear

  35. Yang, H., Cai, X.-C.: Parallel two-grid semismooth Newton-Krylov-Schwarz method for nonlinear complementarity problems. J. Sci. Comput. 47, 258–280 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  36. Yang, H., Prudencio, E., Cai, X.-C.: Fully implicit Lagrange-Newton-Krylov-Schwarz algorithms for boundary control of unsteady incompressible flows. Int. J. Numer. Meth. Eng. 91, 644–665 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang, J., Wang, L.L., Rong, Z.: A prolate-element method for nonlinear PDEs on the sphere. J. Sci. Comput. 47, 73–92 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to express their appreciations to the anonymous reviewers for the invaluable comments that greatly improved the quality of the manuscript. This work was supported in part by NSF grant CCF-1216314 and DOE grant DE-SC0001774. H. Yang was also supported in part by NSFC grants 91330111, 11201137 and 11272352. C. Yang was also supported in part by NSFC grants 61170075 and 91130023

Author information

Authors and Affiliations

  1. College of Mathematics and Econometrics, Hunan University, Hunan, 410082, Changsha, People’s Republic of China

    Haijian Yang

  2. Laboratory of Parallel Software and Computational Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, People’s Republic of China

    Chao Yang

  3. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing, 100190, People’s Republic of China

    Chao Yang

  4. Department of Computer Science, University of Colorado Boulder, Boulder, CO, 80309, USA

    Xiao-Chuan Cai

Authors
  1. Haijian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao-Chuan Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Chuan Cai.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, H., Yang, C. & Cai, XC. Parallel Domain Decomposition Methods with Mixed Order Discretization for Fully Implicit Solution of Tracer Transport Problems on the Cubed-Sphere. J Sci Comput 61, 258–280 (2014). https://doi.org/10.1007/s10915-014-9828-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-014-9828-y

Keywords

Search

Navigation