Remarks on dynamic load balancing of integer loads and integral graphs

https://doi.org/10.1016/j.amc.2013.10.053Get rights and content

Abstract

Cvetković et al. (2010) [3] presented an algorithm for dynamic load balancing in discrete load model based on the eigenvectors of the adjacency matrix of the processor network. In case of integral graphs, whose eigenvalues are integers and whose eigenvector basis may be chosen to consist of integers only, this algorithm executes in integer arithmetic, providing an argument for the use of integral graphs as suitable candidates for processor networks. It is observed here that this algorithm, however, is not capable of balancing all integer load distributions.

After specifying a necessary and sufficient condition for properly managing all integer load distributions, it becomes apparent that the proposed algorithm does not actually depend on the integrality of eigenvectors, but on employing a set of balancing flows carrying a unit load from a fixed vertex to each of the remaining vertices in the graph. Such sets exist for every connected graph and, thus, yield a simple load balancing algorithm that can be executed in integer arithmetic for every processor network.

Introduction

Balancing the work load among processors is important for an efficient use of parallel computers. The parallel algorithms often produce work load dynamically, and its amount per processor may change drastically during the execution. Hence, the total work load needs to be occasionally rebalanced among the neighbouring processors in the network. The common approach for dynamic load balancing is the 2-step model [1], where in the first step a balancing flow, specifying the exact amounts of load that each processor has to send to (or receive from) its neighbouring processors, is calculated, while in the second step, the load elements are migrated accordingly. We further assume that loads are nonnegative integers, that no load is generated or consumed during the balancing process, and that the network is static and homogeneous, meaning that the topology of the network is fixed and that processors have equal computing power, respectively.

Assuming that the global load vector is known, Hu et al. [2] showed that the l2-optimal balancing flow may be computed by using Lagrange multipliers and solving a system of linear equations using the conjugate gradient algorithm. Although the loads are integers, the conjugate gradient algorithm works in floating point arithmetic and the final balancing flow is obtained by rounding the loads to be transferred to the nearest integer.

With the focus on execution in integer arithmetic, Cvetković et al. [3] presented recently in this journal a dynamic load balancing algorithm based on the eigenvectors of the processor network. This algorithm should execute in integer arithmetic, provided that the processor network is an integral graph, whose eigenvalues are integers and whose eigenvector basis may be chosen to consist of integers only. We show in Section 2, however, that this algorithm may result in balancing flows that contain noninteger values, obstructing the focus on integer arithmetic.

In Section 3 we further show that, in order to properly manage all integer load distributions, the approach of Cvetković et al. [3] has to employ a set of balancing flows carrying a unit load from a fixed processor to any other processor in the network (instead of employing the eigenvectors). Such set of flows exist for every connected graph, yielding a fast dynamic load balancing algorithm that may be executed in integer arithmetic for any network. In terms of the l2-norm, it gives suboptimal balancing flows compared to the algorithm of Hu et al. [2], but it may use far less network edges for load migration.

Let G=(V,E) be a weakly connected, directed graph with the processors V={1,,n} and m=|E| edges. Let wiZ be the load of the processor iV and let wZn be the load vector. Let λ=1ni=1nwi be the average load. A vector wZn satisfying wi{λ,λ} represents the uniform load distribution.

Let N{-1,0,+1}n×m be the vertex-edge incidence matrix of G, where the column corresponding to the edge eE contains -1 for the tail and +1 for the head of e, while the remaining entries of this column are 0s. For a vertex iV, let in(i) be the set of edges having i as the head and let out(i) be the set of edges having i as the tail.

Let xZE be a flow on the edges of G. The direction of the flow is given by the directions of edges and the signs of the entries of x: xe>0 denotes a flow in the direction of edge e,eE, xe<0 in the opposite direction. The goal of the load balancing algorithm is, therefore, to find the balancing flow fZE such that for each iV:wi=wi+ein(i)fe-eout(i)fe,or, in other words, such thatw-w=Nf.

Section snippets

Algorithm of Cvetković et al.

A graph is called integral if all eigenvalues of its adjacency matrix are integers. An integral graph, besides integer eigenvalues, has also an eigenbasis consisting of integer eigenvectors (see, e.g., [3, Algorithm 4.1] and [4]). Further, a regular graph has an all-one vector j as the eigenvector of its largest eigenvalue.

It was argued in [Section 4] [3] that regular integral graphs are suitable candidates for processor networks of parallel computers, as it should be possible to perform the

Analyzing and restating the algorithm of Cvetković et al.

In this section we will analyze the approach used in the above algorithm in order to derive a necessary and sufficient condition for it to yield integer balancing flow. After restating the algorithm from a new viewpoint, it will become evident that its work does not depend on the integrality of eigenvectors, but instead on a set of balancing flows carrying a unit of load from a fixed vertex to each of the remaining vertices.

Let B be the n×(n-1) matrix of integral eigenvectors of the regular

Conclusions

Algorithm 2 yields a simple and fast load balancing algorithm working in integer arithmetic that, for each edge of the processor network, reduces to a single scalar multiplication of a column of C corresponding to the edge and the global imbalance vector. However, the price for the simplicity and the speed of the load balancing algorithm is paid by not optimizing the amount of communication necessary to distribute the loads in the network, as the load balancing always uses only the n-1 edges

Acknowledgement

The author thanks Robert Elsässer for helpful discussion of load balancing topics.

References (4)

There are more references available in the full text version of this article.

Supported by the research Grant 174033 of the Serbian Ministry of Science and Education, the research programme P1-0285 and the research project J1-4021 of the Slovenian Agency for Research.

View full text