Skip to main content
SpringerLink
Log in

On the design of two-stage multiprojection methods for distributed memory systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Solving large sparse linear systems, efficiently, on supercomputing infrastructures is a time-consuming component for a wide variety of simulation processes. An effective parallel solver should meet the required specifications, concerning both convergence behavior and scalability. Herewith, a class of two-stage algebraic domain decomposition preconditioning schemes based on the upper Schur complement method is proposed, in order to exploit appropriately distributed memory systems with multicore processors. The design of the method has been focused on homogeneous hybrid parallel systems, i.e., distributed and shared memory systems. However, the proposed method can also be applied to heterogeneous systems, such as cloud infrastructures, or hybrid parallel systems with accelerators, by modifying the workload distribution algorithm and taking into account the different network latencies and bandwidths. The first stage of the proposed schemes is related to the assignment of the subdomains among the workstations of the distributed system, whereas the second stage concerns the further redistribution of the subdomains to each core of a processor. The proposed method utilizes multiprojection techniques, based on semi-aggregated subdomains, leading to improved convergence behavior as the number of subdomains increases. Moreover, a subspace compression technique is used, in order to improve the performance of the preprocessing phase and reduce the memory requirements of the proposed scheme. The preconditioning schemes were combined with a parallel Krylov subspace method, i.e., the parallel preconditioned GMRES(m) method. The convergence behavior, the performance and the scalability of the proposed preconditioning schemes are examined and compared to existing state-of-the-art methods, by conducting several numerical experiments on supercomputing infrastructures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Agullo E, Giraud L, Guermouche A, Haidar A, Roman J (2013) Parallel algebraic domain decomposition solver for the solution of augmented systems. Adv Eng Softw 60–61:23–30. https://doi.org/10.1016/j.advengsoft.2012.07.004

    Article  Google Scholar 

  2. Axelsson O (1996) Iterative solution methods. Cambridge University Press, London

    MATH  Google Scholar 

  3. Bank R, Falgout R, Jones T, Manteuffel TA, McCormick SF, Ruge JW (2015) Algebraic multigrid domain and range decomposition (amg-dd/amg-rd). SIAM J Sci Comput 37(5):S113–S136

    MathSciNet  MATH  Google Scholar 

  4. Bank RE, Smith RK (2002) An algebraic multilevel multigraph algorithm. SIAM J Sci Comput 23(5):1572–1592

    MathSciNet  MATH  Google Scholar 

  5. Benzi M (2002) Preconditioning techniques for large linear systems: a survey. J Comput Phys 182(2):418–477

    MathSciNet  MATH  Google Scholar 

  6. Bramble JH, Pasciak JE, Schatz AH (1986) The construction of preconditioners for elliptic problems by substructuring. i. Math Comput 47(175):103–134

    MathSciNet  MATH  Google Scholar 

  7. Brezina M, Vaněk P (1999) A black-box iterative solver based on a two-level schwarz method. Computing 63(3):233–263

    MathSciNet  MATH  Google Scholar 

  8. Cai XC, Sarkis M (1999) A restricted additive schwarz preconditioner for general sparse linear systems. SIAM J Sci Comput 21(2):792–797

    MathSciNet  MATH  Google Scholar 

  9. Chan TF, Mathew TP (1994) Domain decomposition algorithms. Acta Numer 3:61–143

    MathSciNet  MATH  Google Scholar 

  10. Chevalier C, Pellegrini F (2008) Pt-scotch: a tool for efficient parallel graph ordering. Parallel Comput 34(6–8):318–331

    MathSciNet  Google Scholar 

  11. Davis TA (2004) Algorithm 832: umfpack, anunsymmetricpattern multifrontal method with a column preordering strategy. ACM Trans Math Softw 30(2):196–199

    MathSciNet  Google Scholar 

  12. Davis TA (2006) Direct methods for sparse linear systems, vol 2. SIAM, New Delhi

    MATH  Google Scholar 

  13. Erlangga YA, Nabben R (2009) Algebraic multilevel krylov methods. SIAM J Sci Comput 31(5):3417–3437

    MathSciNet  MATH  Google Scholar 

  14. Ferronato M, Janna C, Pini G (2014) A generalized Block FSAI preconditioner for nonsymmetric linear systems. J Comput Appl Math 256:230–241

    MathSciNet  MATH  Google Scholar 

  15. Filelis-Papadopoulos CK, Gravvanis GA (2016) A class of generic factored and multi-level recursive approximate inverse techniques for solving general sparse systems. Eng Comput 33(1):74–99

    Google Scholar 

  16. Giraud L, Haidar A, Saad Y (2010) Sparse approximations of the schur complement for parallel algebraic hybrid linear solvers in 3D. Numer Math Theory Methods Appl 3(3):276–294

    MathSciNet  MATH  Google Scholar 

  17. Grote MJ, Huckle T (1997) Parallel preconditioning with sparse approximate inverses. SIAM J Sci Comput 18(3):838–853

    MathSciNet  MATH  Google Scholar 

  18. Have P, Masson R, Nataf F, Szydlarski M, Xiang H, Zhao T (2013) Algebraic domain decomposition methods for highly heterogeneous problems. SIAM J Sci Comput 35(3):C284–C302

    MathSciNet  MATH  Google Scholar 

  19. Janna C, Ferronato M, Gambolati G (2013) Enhanced block fsai preconditioning using domain decomposition techniques. SIAM J Sci Comput 35(5):S229–S249

    MathSciNet  MATH  Google Scholar 

  20. Karypis G, Kumar V (1998) A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J Sci Comput 20(1):359–392

    MathSciNet  MATH  Google Scholar 

  21. Karypis G, Kumar V (1998) Multilevel k-way partitioning scheme for irregular graphs. J Parallel Distrib Comput 48(1):96–129

    MATH  Google Scholar 

  22. Li R, Saad Y (2017) Low-rank correction methods for algebraic domain decomposition preconditioners. SIAM J Matrix Anal Appl 38(3):807–828

    MathSciNet  MATH  Google Scholar 

  23. Li Z, Saad Y (2006) Schurras: a restricted version of the overlapping schur complement preconditioner. SIAM J Sci Comput 27(5):1787–1801

    MathSciNet  MATH  Google Scholar 

  24. Li Z, Saad Y, Sosonkina M (2003) pARMS: a parallel version of the algebraic recursive multilevel solver. Numer Linear Algebra Appl 10(5–6):485–509

    MathSciNet  MATH  Google Scholar 

  25. Mandel J (1990) Two-level domain decomposition preconditioning for the p-version finite element method in three dimensions. Int J Numer Methods Eng 29(5):1095–1108

    MathSciNet  MATH  Google Scholar 

  26. Manguoglu M (2012) Parallel solution of sparse linear systems. In: Berry M et al (eds) High-performance scientific computing. Springer, Berlin, pp 171–184

    Google Scholar 

  27. Mitchell WF (1997) A parallel multigrid method using the full domain partition. Electron Trans Numer Anal 6:224–233

    MathSciNet  MATH  Google Scholar 

  28. Moutafis BE, Filelis-Papadopoulos CK, Gravvanis GA (2017) Parallel multi-projection preconditioned methods based on semi-aggregation techniques. J Comput Sci 22:45–53

    MathSciNet  MATH  Google Scholar 

  29. Moutafis BE, Filelis-Papadopoulos CK, Gravvanis GA (2017) Parallel multi-projection preconditioned methods based on subspace compression. Math Problems Eng 2017, 2580820. https://doi.org/10.1155/2017/2580820

    Article  MATH  Google Scholar 

  30. Moutafis BE, Filelis-Papadopoulos CK, Gravvanis GA (2018) Parallel schur complement techniques based on multiprojection methods. SIAM J Sci Comput 40(4):C634–C654

    MathSciNet  MATH  Google Scholar 

  31. Nataf F, Xiang H, Dolean V, Spillane N (2011) A coarse space construction based on local dirichlet-to-neumann maps. SIAM J Sci Comput 33(4):1623–1642

    MathSciNet  MATH  Google Scholar 

  32. Nicolaides RA (1987) Deflation of conjugate gradients with applications to boundary value problems. SIAM J Numer Anal 24(2):355–365

    MathSciNet  MATH  Google Scholar 

  33. Pellegrini F, Roman J (1996) Scotch: a software package for static mapping by dual recursive bipartitioning of process and architecture graphs. In: Liddell H, Colbrook A, Hertzberger B, Sloot P (eds) International Conference on High-performance Computing and Networking. Springer, Berlin, pp 493–498

    Google Scholar 

  34. Saad Y (2003) Iterative methods for sparse linear systems, 2nd edn. Society for industrial and applied mathematics. SIAM, Philadelphia

    MATH  Google Scholar 

  35. Schenk O, Gärtner K (2004) Solving unsymmetric sparse systems of linear equations with pardiso. Future Gener Comput Syst 20(3):475–487

    MATH  Google Scholar 

  36. Smith B, Bjorstad P, Gropp WD, Gropp W (2004) Domain decomposition: parallel multilevel methods for elliptic partial differential equations. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  37. Tang JM, Nabben R, Vuik C, Erlangga YA (2009) Comparison of two-level preconditioners derived from deflation, domain decomposition and multigrid methods. J Sci Comput 39(3):340–370

    MathSciNet  MATH  Google Scholar 

  38. Toselli A, Widlund OB (2005) Domain decomposition methods: algorithms and theory, vol 34. Springer, Berlin

    MATH  Google Scholar 

  39. Tselepidis N, Filelis-Papadopoulos C, Gravvanis G (2018) Distributed algebraic tearing and interconnecting techniques. Numer Algorithms 82:1–34

    MathSciNet  MATH  Google Scholar 

  40. Van der Vorst HA (2003) Iterative Krylov methods for large linear systems, vol 13. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  41. Xi Y, Li R, Saad Y (2016) An algebraic multilevel preconditioner with low-rank corrections for sparse symmetric matrices. SIAM J Matrix Anal Appl 37(1):235–259

    MathSciNet  MATH  Google Scholar 

  42. Xiang H, Nataf F (2014) Two-level algebraic domain decomposition preconditioners using jacobi-schwarz smoother and adaptive coarse grid corrections. J Comput Appl Math 261:1–13

    MathSciNet  MATH  Google Scholar 

  43. Zhang J (2000) On preconditioning schur complement and schur complement preconditioning. Electron Trans Numer Anal 10:115–130

    MathSciNet  MATH  Google Scholar 

  44. Zhu Y, Sameh AH (2017) PSPIKE+: a family of parallel hybrid sparse linear system solvers. J Comput Appl Math 311:682–703

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The research work of Byron E. Moutafis, as a Ph.D. candidate, was funded by the General Secretariat for Research and Technology (GSRT) and the Hellenic Foundation for Research and Innovation (HFRI) (Grant Code: 1609). The authors acknowledge the Greek Research and Technology Network (GRNET) for the provision of the National HPC facility ARIS under project PR006053-ScaleSciCompIII.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, School of Engineering, University Campus, 67100, Kimmeria, Xanthi, Greece

    B. E. Moutafis, G. A. Gravvanis & C. K. Filelis-Papadopoulos

Authors
  1. B. E. Moutafis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. A. Gravvanis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. K. Filelis-Papadopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. A. Gravvanis.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moutafis, B.E., Gravvanis, G.A. & Filelis-Papadopoulos, C.K. On the design of two-stage multiprojection methods for distributed memory systems. J Supercomput 76, 9063–9094 (2020). https://doi.org/10.1007/s11227-020-03201-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-020-03201-5

Keywords

Search

Navigation