Skip to main content

Advertisement

Springer Nature Link
Log in

Scalable implementation of multigrid methods using partial semi-aggregation of coarse grids

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Multigrid methods are efficient and fast algorithms for solving elliptic equations. However, they suffer from the degradation of parallel efficiency on coarser levels: communication costs are much higher than computational costs when the number of computing core becomes massive. This study presents a coarse-grid partial semi-aggregation (CGPSA) method, where coarse grids in each direction are independently aggregated step-by-step across multiple levels. This multilevel aggregation provides a hierarchical communication structure. Thus, communication overheads are distributed into multiple levels and computational workloads are processed by multiple processes on coarser levels. Independent coarse-grid aggregation along each dimension also enhances the flexibility of multigrid method; it can be applied to the problem with a non-cubic geometry and anisotropic sizes of grids and parallel processes. Benchmark results on a large-scale cluster (up to 65k cores) are provided with a detailed performance profiling. The results show the improvement of parallel performance and scalability with the CGPSA method, which focuses on reducing communication overheads at coarser levels. They also show that the changes in the order of semi-aggregation levels can further improve performance, which implies that the proposed CGPSA method presents further scope for performance optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Pan X, Kim K, Lee C, Choi JI (2017) Fully decoupled monolithic projection method for natural convection problems. J Comput Phys 334:582

    Article  MathSciNet  Google Scholar 

  2. Zhu X, Phillips E, Spandan V, Donners J, Ruetsch G, Romero J, Ostilla-Mónico R, Yang Y, Lohse D, Verzicco R et al (2018) AFi D-GPU: a versatile Navier–Stokes solver for wall-bounded turbulent flows on GPU clusters. Comput Phys Commun 229:199

    Article  Google Scholar 

  3. Knight P, Thyagaraja A, Edwards T, Hein J, Romanelli M, McClements K (2012) CENTORI: a global toroidal electromagnetic two-fluid plasma turbulence code. Comput Phys Commun 183(11):2346

    Article  Google Scholar 

  4. Kourtzanidis K, Rogier F, Boeuf JP (2015) ADI-FDTD modeling of microwave plasma discharges in air towards fully three-dimensional simulations. Comput Phys Commun 195:49

    Article  MathSciNet  Google Scholar 

  5. Tay WC, Tan EL (2014) Pentadiagonal alternating-direction-implicit finite-difference time-domain method for two-dimensional Schrödinger equation. Comput Phys Commun 185(7):1886

    Article  MathSciNet  Google Scholar 

  6. Li LZ, Sun HW, Tam SC (2015) A spatial sixth-order alternating direction implicit method for two-dimensional cubic nonlinear Schrödinger equations. Comput Phys Commun 187:38

    Article  MathSciNet  Google Scholar 

  7. Sak H, Özekici S, Bodurog Í (2007) Parallel computing in Asian option pricing. Parallel Comput 33(2):92

    Article  MathSciNet  Google Scholar 

  8. Egloff D (2012) GPU computing gems jade edition. Elsevier, Amsterdam, pp 309–322

    Book  Google Scholar 

  9. Hackbusch W (1985) Multi-grid methods and applications. Springer, Berlin

    Book  Google Scholar 

  10. Wesseling P (2004) An introduction to multigrid methods. R.T, Edwards

  11. Dai R, Lin P, Zhang J (2019) An EXCMG accelerated multiscale multigrid computation for 3D Poisson equation. Comput Math Appl 77(8):2051

    Article  MathSciNet  Google Scholar 

  12. Arrarás A, Gaspar F, Portero L, Rodrigo C (2019) Geometric multigrid methods for Darcy–Forchheimer flow in fractured porous media. Comput Math Appl 78(9):3139

    Article  MathSciNet  Google Scholar 

  13. Zolfaghari H, Becsek B, Nestola MG, Sawyer WB, Krause R, Obrist D (2019) High-order accurate simulation of incompressible turbulent flows on many parallel GPUs of a hybrid-node supercomputer. Comput Phys Commun 244:132

    Article  MathSciNet  Google Scholar 

  14. Chow W, Falgout RD, Hu JJ, Tuminaro RS, Yang UM (2005) In: Heroux MA, Raghavan P, Simon HD (eds) Parallel processing for scientific computing. SIAM Publications, Philadelphia

  15. Kang K (2015) Scalable implementation of the parallel multigrid method on massively parallel computers. Comput Math Appl 70(11):2701

    Article  MathSciNet  Google Scholar 

  16. Esmaily M, Jofre L, Mani A, Iaccarino G (2018) A scalable geometric multigrid solver for nonsymmetric elliptic systems with application to variable-density flows. J Comput Phys 357:142

    Article  MathSciNet  Google Scholar 

  17. Nakajima K (2012) In: Proceedings of the 2012 International Workshop on Programming Models and Applications for Multicores and Manycores. Association for Computing Machinery, New York. PMAM, pp 93–102

  18. Nakajima K (2012) In: Proceedings of the 2012 IEEE International Conference on Cluster Computing Workshops. IEEE Computer Society, USA, CLUSTERW 2012, pp 199–206

  19. Kang KS (2010) Parallelization of the multigrid method on high performance computers. Technical Report IPP-Report 5/123, Max-Planck-Institut für Plasmaphysik

  20. Reiter S, Vogel A, Heppner I, Rupp M, Wittum G (2013) A massively parallel geometric multigrid solver on hierarchically distributed grids. Comput Vis Sci 16:151

    Article  Google Scholar 

  21. Nakajima K (2014) In: Proceedings of the 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS), pp. 25–32

  22. Nakajima K, Gerofi B, Ishikawa Y, Horikoshi M (2019) In: Proceedings of the 2019 IEEE/ACM 10th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScalA), pp 52–61

  23. Zhang J (2002) Multigrid method and fourth-order compact scheme for 2D Poisson equation with unequal mesh-size discretization. J Comput Phys 179:170

    Article  Google Scholar 

  24. Ge Y (2010) Multigrid method and fourth-order compact difference discretization scheme with unequal meshsizes for 3D Poisson equation. J Comput Phys 229:6381

    Article  MathSciNet  Google Scholar 

  25. Cao F, Ge Y, Sun HW (2017) Partial semi-coarsening multigrid method based on the HOC scheme on nonuniform grids for the convection-diffusion problems. Int J Comput Math 94(12):2356

    Article  MathSciNet  Google Scholar 

  26. Li ZH, Chen L, Tao WQ (2017) A parallel scalable multigrid method and HOC scheme for anisotropy elliptic problems. Numer Heat Transf B Fundam 71(4):346

    Article  Google Scholar 

  27. Kim KH, Kang JH, Pan X, Choi JI (2021) PaScaLTDMA: a library of parallel and scalable solvers for massive tridiagonal systems. Comput Phys Commun 260:107722

    Article  Google Scholar 

  28. Qiang J (2017) A fast parallel 3D Poisson solver with longitudinal periodic and transverse open boundary conditions for space-charge simulations. Comput Phys Commun 219:255

    Article  Google Scholar 

  29. Malagrón-Romero A, Luque A (2018) A domain-decomposition method to implement electrostatic free boundary conditions in the radial direction for electric discharges. Comput Phys Commun 225:114

    Article  Google Scholar 

  30. Kang JH, Ryu J, Ryu H (2021) Exploring the behaviors of electrode-driven Si quantum dot systems: from charge control to qubit operations. Nanoscale 13:332

    Article  Google Scholar 

  31. Moon S, Kim WT, Ostriker EC (2019) A fast Poisson solver of second-order accuracy for isolated systems in three-dimensional cartesian and cylindrical coordinates. Astrophys J Suppl Ser 241(2):24

    Article  Google Scholar 

  32. Kang JH, Hwang J, Sung HJ, Ryu H (2021). J Supercomput. https://doi.org/10.1007/s11227-021-03642-6

  33. Lozano-Durán A, Jiménez J (2014) Effect of the computational domain on direct simulations of turbulent channels up to Re\(_{\tau }{\approx }\) 4200. Phys Fluids 26:011702

    Article  Google Scholar 

  34. Lee M, Moser RD (2015) Direct numerical simulation of turbulent channel flow up to \(Re_{\tau } \approx 5200\). J Fluid Mech 774:395

    Article  Google Scholar 

  35. Nakajima K (2002) Parallel multilevel iterative linear solvers with unstructured adaptive grids for simulations in earth science. Concurr Comput Pract Exp 14(6–7):483

    Article  Google Scholar 

  36. Kang JH (2021) Poisson equation parallel solver: multigrid method. https://doi.org/10.5281/ZENODO.4777391

Download references

Acknowledgements

This research was supported by the Korea Institute of Science and Technology Information (KISTI): Institutional R&D program (K-20-L02-C09) and the grant from the National Research Foundation of Korea (NRF2020M3H6A1084853) funded by the Korean government (MSIT). The Nurion supercomputer in KISTI is extensively utilized to carry out this work (KSC-2019-CRE-0180).

Author information

Authors and Affiliations

  1. National Institute of Supercomputing and Networking, Korea Institute of Science and Technology Information, Daejeon, 34141, Republic of Korea

    Ji-Hoon Kang

Authors
  1. Ji-Hoon Kang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ji-Hoon Kang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, JH. Scalable implementation of multigrid methods using partial semi-aggregation of coarse grids. J Supercomput 78, 833–851 (2022). https://doi.org/10.1007/s11227-021-03912-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03912-3

Keywords