A parallel multigrid solver for 3D convection and convection–diffusion problems☆
Introduction
The convergence properties of multigrid algorithms are defined by two factors: (1) the smoothing rate, which describes the reduction of high-frequency error components, and (2) the quality of the coarse-grid correction, which is responsible for the dumping of smooth error components. In elliptic problems, all the smooth fine-grid components are well approximated on the coarse grid built by standard (full) coarsening. In non-elliptic problems, however, some fine-grid characteristic components that are much smoother in the characteristic direction than in other directions, cannot be approximated with standard multigrid methods (see [1], [2], [3], [6]).
Several approaches aimed at curing the characteristic-component problem have been studied in literature. These approaches fall into two categories: (1) development of a suitable relaxation scheme to eliminate not only high-frequency error components but the characteristic error components as well; (2) devising an adjusted coarse-grid operator to approximate well the fine-grid characteristic error components.
In many practical problems appearing in computational fluid dynamics (CFD), the non-elliptic part is represented by convection. For convection, the most efficient and comprehensive relaxation is downstream marching. If the target discretization is a stable upwind scheme, the downstream marching reduces all (high-frequency and smooth) error components, solving a non-linear convection equation in just a few sweeps (a single downstream sweep provides the exact solution to a linearized problem). Incomplete LU (ILU) decomposition methods act similarly, given a suitable ordering [5]. The downstream marching technique was successfully applied in solving many CFD problems associated with non-recirculating flows (see, e.g., [3]). However, if the discretization is not fully upwind (e.g., upwind biased) the downstream marching in its pure form is not viable. One of the most efficient (also marching type) alternatives often applied to the schemes that cannot be directly marched is a defect-correction method (see, e.g., [18], [23], [24]). Usually the efficiency of these methods is quite satisfactory. Sometimes, however, the convergence rate of the defect-correction method is grid dependent (see [7], [8]). Another, very important, drawback associated with all marching and ILU methods is a low-parallel grade, because the efficiency of these methods is essentially based on the correctness of the sequential marching order.
In methods belonging to the second category, most of the operations can be performed in parallel. These methods are much more attractive for massive parallel computing. The necessary requirements for coarse-grid operators used in second-category methods were formulated in [27]. Among the options available in conjunction with full coarsening are Galerkin coarsening [28], matrix-dependent operators [5], and corrected coarse-grid operators [11]. Analysis in [27] showed that all these methods have certain drawbacks.
Another way to construct an appropriate coarse-grid operator is to employ semicoarsening [6]. The multigrid method evaluated in this paper uses semicoarsening together with a well-balanced correction of discrete operators to maintain the same cross-characteristic interaction (CCI) on all the grids. The relaxation scheme employed in this algorithm is a four-color plane-implicit scheme enabling a very efficient parallel implementation. The resulting algorithm is an efficient highly parallel method for solving the three-dimensional (3D) convection operator defined on cell-centered grids with stretching.
When studying the optimal parallel implementation of a numerical algorithm, one should consider both the numerical and parallel properties. The best approach in a sequential setting may not be the optimal one on a parallel computer. The multigrid method proposed in this paper exhibits a very fast convergence rate but the granularity of its smoother (four-color relaxation) is finer than that of other common smoothers as zebra or damped Jacobi. In order to improve the granularity of the solver, we have studied a hybrid smoother (HS) that uses a four-color, zebra or damped Jacobi update depending on the level. As we will show, although this smoother degrades the convergence properties of the original method, it improves the execution time of the multigrid cycle in a parallel setting, and so becomes a trade-off between numerical and architectural properties.
Section 2 formulates the model problem for the convection equation. The 3D discretizations and the difficulties encountered in multigrid methods solving the convection operator are explained in Section 3. Section 4 presents the multigrid cycle for the full-dimensional problem. Section 5 includes numerical results confirming the efficient solution of the convection equation on uniform and stretched grids. Section 6 demonstrates an extension of the tested method to the convection–diffusion equation. The parallel properties of the MPI implementation of the code on SGI Origin 2000 and Cray T3E systems are discussed in Section 7. Finally, the main conclusions and future research directions are presented in Section 8. More detailed discussions related to this paper material and some additional numerical tests can be found in [14].
Section snippets
Convection equation
The model problem studied in this section is the 3D constant-coefficient convection equationwhere is a given vector and . The solution U(x,y,z) is a differentiable function defined on the unit cube (x,y,z)∈[0,1]×[0,1]×[0,1]. Let ty=a2/a1 and tz=a3/a1 be non-alignment parameters. For simplicity, we assume a1⩾a2,a3⩾0 and, therefore, 1⩾ty,tz⩾0.
Eq. (2.1) can be rewritten aswhere is a variable along the
Rectangular-grid discretizations
The discretization grids considered in the paper are 3D Cartesian rectangular grids with aspect ratios my=hx/hy and mz=hx/hz, where hx, hy and hz are the mesh sizes in the x, y and z directions, respectively. The target (finest) grid is uniform (hx=hy=hz=h). The basic discretization to the problem (2.2) on a rectangular grid is obtained from the low-dimensional prototype discretization (2.4) by replacing function values at the ghost points (points with fractional indexes) by weighted averages
The multigrid method
The proposed multigrid method for solving the convection equation employs semicoarsening and narrow coarse-grid discretization schemes supplied with explicit terms (which are discrete approximations to hy∂yy and hz∂zz with suitable coefficients) to maintain on all the grids the same uniform CCI. This construction ensures that all the characteristic error components are eliminated fully by the coarse-grid correction. The non-characteristic error components must be reduced in relaxation.
Numerical results
The inflow boundary conditions for the test problems were chosen so that the function U(x,y,z)=cos(ω(y+z−(ty+tz)x)) is the exact continuous solution of the homogeneous (fi1,i2,i3=0) problem (2.2). The initial approximation was interpolated from the solution on the previous coarse grid.
The frequencies ω=8π for a 643 grid and ω=16π for a 1283 grid were chosen to reduce the total computational time exploiting periodicity and to provide a reasonable accuracy in approximating the true solution of
Efficient solution of the convection–diffusion operator
In this section we will study the 3D constant-coefficient convection–diffusion equationwhere is a given vector and ϵ is a positive scalar. The solution U(x,y,z) is a differentiable function defined on the unit square (x,y,z)∈[0,1]×[0,1]×[0,1].
Eq. (6.1) can be rewritten asand with the additional streamwise dissipation aswhere , and is a variable
Parallel implementation
The main advantage of the multigrid algorithm proposed in this paper is its parallel potential. We now describe a parallel implementation of the algorithm, based on MPI, and its efficiency on two different parallel systems; a Cray T3E-900 (T3E) and an SGI Origin 2000 (O2K).
The test problem chosen in the experiments carried out in this section is the convection–diffusion Eq. (6.3) with boundary conditions (6.4), so that g(y,z)=cos(ω(y+z)), and Fi1,i2,i3=0. We have selected frequencies ω=8π and 16
Conclusions and future work
The combination of semicoarsening, a four-color plane-implicit smoother and the introduction of explicit CCI terms in the discretization of all grids yields an efficient highly parallel multigrid solver for the convection–diffusion equation with fast grid-independent convergence rates for any angle of non-alignment. This solver permits the parallel solution of a convective process that is sequential in nature.
We have opted to employ a 1D grid partitioning in the semicoarsened direction. This
Acknowledgements
We would like to thank Ciemat, CSC (Centro de Supercomputación Complutense), ICASE and the Computational Modeling and Simulation Branch at the NASA Langley Research Center for providing access to the parallel computers that have been used in this research.
References (28)
- et al.
On multigrid solution of high-reynolds incompressible entering flow
J. Comput. Phys.
(1992) Matrix-dependent prolongations and restrictions in a blackbox multigrid solver
J. Comput. Appl. Math.
(1990)Black box multigrid for nonsymmetric problems
Appl. Math. Comput.
(1983)- et al.
Unified framework for the parallelization of divide and conquer based tridiagonal systems
Parallel Comput.
(1997) - et al.
Some aspects about the scalability of scientific applications on parallel architectures
Parallel Comput.
(1996) - et al.
Algorithm for solving tridiagonal matrix problems in parallel
Parallel Comput.
(1995) - et al.
Multigrid line smoothers for higher order upwind discretizations of convection-dominated problems
J. Comput. Phys.
(1998) Parallelization of pipelined algorithms for sets of linear banded systems
J. Parallel Distributed Comput.
(1999)- et al.
Parallel multigrid for anisotropic elliptic equations
J. Parallel Distributed Comput.
(2001) - A. Brandt, Multigrid solvers for non-elliptic and singular-perturbation steady-state problems, The Weizmann Institute...
Multigrid solvers for nonaligned sonic flows
SIAM J. Sci. Comp.
Parallel Computer Architecture. A Hardware/Software Approach
Half-space analysis of the defect-correction method for fromm discretization of convection
SIAM J. Sci. Comp.
Cited by (7)
An extremum-preserving finite volume scheme for convection-diffusion equation on general meshes
2020, Applied Mathematics and ComputationCitation Excerpt :The convection-diffusion equation has many applications, for example the gas dynamics, the process of groundwater transport of a solute in porous media [1–3].
Fast and high accuracy multiscale multigrid method with multiple coarse grid updating strategy for the 3D convection–diffusion equation
2013, Computers and Mathematics with ApplicationsCitation Excerpt :Gupta and Zhang realized parallelization and vectorization by using four colors for 19 point scheme [8] and using two colors for 15 point scheme [9]. For other parallel computations of the 3D convection–diffusion equation, readers are referred to [10–12]. Up to now, fourth order compact (FOC) schemes for solving the 3D convection–diffusion equation have been studied extensively in the literature.
Parallel computing as a vehicle for engineering design of complex functional surfaces
2011, Advances in Engineering SoftwareCitation Excerpt :While the advantage offered by parallel multigrid methods are easily comprehended, there are potential drawbacks embedded within the approach; in particular, concerning the high communication costs relative to computational costs when computations are performed on coarse grid levels. However, this can be alleviated in different ways [21] – in the present study, the coarsest grid level is selected such that the cost of communication to computation ratio stays within a reasonable range [22]. The problem of interest and associated mathematical model are described in Section 2.
Fast and robust sixth-order multigrid computation for the three-dimensional convectiondiffusion equation
2010, Journal of Computational and Applied MathematicsA parallel multigrid solver for viscous flows on anisotropic structured grids
2003, Parallel ComputingCitation Excerpt :From an implementational point of view, this is by far the best scheme that can be considered, since it avoids the programming effort and the overheads that a parallel plane solver introduces into the code. These considerations have been employed for example in [9,20] to parallelize a robust multigrid algorithm for the anisotropic diffusion and advection equations respectively. Nevertheless, although 1-D decompositions have no need for a parallel plane smoother, they also have some drawbacks.
On quantitative analysis methods for multigrid solutions
2005, SIAM Journal on Scientific Computing
- ☆
This research was supported by the National Aeronautics and Space Administration under NASA Contract No. NAS1-97046 while the authors were in residence at the Institute for Computer Applications in Science and Engineering (ICASE), NASA Langley Research Center, Hampton, VA 23681-2199. Ignacio M. Llorente and Manuel Prieto-Matı́as were also supported in part by the Spanish research grant CICYT TIC 99/0474 and the US-Spain Joint Commission for Scientific and Technological Cooperation.