Elsevier

Parallel Computing

Volume 26, Issue 12, November 2000, Pages 1661-1680
Parallel Computing

Partitioning strategies for structured multiblock grids

https://doi.org/10.1016/S0167-8191(00)00044-2Get rights and content

Abstract

A framework is presented for partitioning of multiblock grids used in data parallel applications. It includes partitioning strategies found in the literature, as well as new algorithms proposed here. In particular, a multilevel graph-partitioning strategy – specifically designed for structured composite grids – is proposed. Different partitioning strategies are compared in two case studies involving multiblock grids. One of the applications is a profile of a multi-element wing from Airbus A310. The partition properties from the different strategies and algorithms depend very much on the number of subgrids and their sizes as well as on the number of processors. Results demonstrate that existing strategies behave well in some cases while the new multilevel graph-partitioning strategy gives an overall good performance.

Introduction

Numerical solution of partial differential equations with finite difference methods has proven to be efficient and accurate. Finite difference methods require a structured discretization of the domain. However, single structured grids are not sufficient in many realistic applications and composite grid methods must be used. A composite grid is a union of structured grids. The computational domain is decomposed into logically rectangular subdomains. Each subdomain is discretized into a subgrid and constitutes a component of the composite grid. The different subgrids interact with each other via interpolation. A common variant is the so-called multiblock grid where the subgrids meet edge-to-edge (see Fig. 1). These grids are frequently used in, for example, aerodynamical computations. Even though the grids meet edge-to-edge, they can have irregular connections, i.e. an edge of a grid can connect to several other grid edges. These grids are also referred to as irregularly coupled regular meshes.

The issue in this paper is the partitioning of structured multiblock grids used in data parallel numerical algorithms, such as explicit time integration methods. To improve the efficiency and to simplify the data structures, the partitions must be rectangular blocks. Partitioning a structured multiblock grid is still non-trivial due to the different sizes of the subgrids and the irregular data dependencies in the interpolation between subgrids. We have developed new partitioning algorithms, based on graph-partitioning methods, that consider both the different sizes and the irregular couplings between the subgrids. We compare our algorithms with some previous methods and show that our approach is better in many cases.

The outline of the paper is as follows. In Section 2, we talk about related work in the field. In Section 3, we formulate a general partitioning strategy for multiblock grids. From this strategy, we identify four important special cases which we discuss in detail. One of the special cases includes our new partitioning approach; the other three are already known. In Section 4, we evaluate the four partitioning methods for the simple example shown in Fig. 1, and for a real-life application used in industry, a wing profile from Airbus. Finally, in Section 5, we discuss and summarize the results.

Section snippets

Related work

Partitioning a single structured grid is straightforward for data parallel applications and much previous work treats this problem (see for example [14], [18], [26]). A natural extension to composite grids is to apply the single grid partitioning techniques to one subgrid at a time. Another natural strategy is to distribute different subgrids to different processors. Composite grids exhibit both coarse-grain parallelism at the subgrid level, i.e. computing on different subgrids in parallel, and

General approach

We formulate a general partitioning strategy for composite grids which includes the previous partitioning strategies discussed as special cases.

Composite grid partitioning strategy:

Cluster the subgrids and the processors, respectively, in groups. Distribute the groups of subgrids over the groups of processors. Within a processor group, partition the corresponding subgrids over the processors in that group.

Within the framework set by this strategy, we have four degrees of freedom: (i) how to

Applications

In this section, we experimentally compare the partitioning strategies. As case studies, we partition two completely different multiblock grids. Unfortunately, we only had access to a flow solver for the first example. Thus, for the second application, we only do a dry partitioning and compare the number of communication points edge-cut, the load imbalance ratio Λ, and the number of blocks as a function of the number of processors using the different partitioning strategies discussed in Section

Conclusions

We have outlined a general strategy for partitioning multiblock grids, irregularly coupled structured meshes. Within the framework set by the general strategy, we have identified four important special cases: block, single, graph, and cluster partitioning strategies. For the cluster strategy, we have given an optimal processor clustering algorithm. This algorithm can also be used in HPF-2 to define optimal processors subspaces. For the graph strategy, we have developed a flexible multilevel

Acknowledgements

This work was completed while the author was at the Department of Scientific Computing, Uppsala University, Sweden. The numerical experiments on the Cray T3D were performed at the Edinburgh Parallel Computing Centre supported by the Training and Research on Advanced Computing systems (TRACS) program.

The grid around the Airbus A310 wing profile was used in a European Garteur cooperation. The topology file and the plot of the topology of the grid was provided by Saab in Linköping, Sweden. Thanks

References (29)

  • Y.P. Chien, F. Carpenter, A. Ecer, H.U. Akay, Load balancing for parallel computation of fluid dynamics problems, in:...
  • R. Enander, Grid patching and residual smoothing for computations of steady state solutions of first order hyperbolic...
  • S.J. Fink, S.B. Baden, Runtime support for multi-tier programming of block-structured applications on SMP clusters, in:...
  • B. Hendrickson, R. Leland, A multilevel algorithm for partitioning graphs, in: Proceedings of the Supercomputing'95,...
  • Cited by (34)

    • Spatial parallel disturbance region update method with OpenMP for steady compressible flows

      2022, Computer Physics Communications
      Citation Excerpt :

      For example, Janus [11] introduced the concept of diagonal index planes into implicit factored time-marching schemes, thereby making the schemes easy to parallelize in terms of the diagonal plane. In general, the coarse-grained parallelism is more popular because of its good scalability [9], and the fine-grained one can be a complemental way to assist its coarse-grained counterpart in pursuit of maximum speed-up. All programming paradigms for central processing units provide the capability to implement the coarse-grained parallelism, such as the message-passing interface (MPI) characterized by distributed memory [12] and the open multi-processing (OpenMP) characterized by shared memory [13].

    • Structured multi-block grid partitioning using balanced cut trees

      2020, Journal of Parallel and Distributed Computing
      Citation Excerpt :

      Obviously, such partitioning tools are barely applicable to generate structured partitions, since they often create heavily jagged partitions that can hardly be represented by a reasonable number of structured subpartitions. Nevertheless, graph partitioning approaches have been applied to structured multi-block grids [20]. Several partitioning algorithms in the literature, e.g., [1,3,16,18,23,25], recursively bisect the longest edges of the largest partition to generate smaller structured partitions that basically allow load balanced distributions to be found.

    • Efficient parallel implementation of large scale 3D structured grid CFD applications on the Tianhe-1A supercomputer

      2013, Computers and Fluids
      Citation Excerpt :

      Post-processing: Analyze and visualize the results of flow simulation and give corresponding report. To meet requirements for parallel computation, the traditional domain decomposition is employed that divides the original problem into multiple tasks [18–20]. A given grid data as well as corresponding boundary conditions must therefore be divided into a lot of blocks, or sub-domains.

    • A 2D block-structured mesh partitioner for accurate flow simulations on non-rectangular geometries

      2011, Computers and Fluids
      Citation Excerpt :

      These geometric techniques are used in elsA software [7] which is devoted to compressible flows around complex geometries. We can also cite the works of Rantakokko [8] who proposes a framework for partitioning composite grids. In our opinion, his more interesting approach consists in a graph strategy applied at the block level instead of the node level (block refinement is also proposed).

    • A Grid-based Flow Analysis and Investigation of Load Balance in Heterogeneous Computing Environment

      2004, Parallel Computational Fluid Dynamics 2003: Advanced Numerical Methods, Software and Applications
    View all citing articles on Scopus
    View full text