Cooperative differential evolution with fast variable interdependence learning and cross-cluster mutation

Highlights

• We propose a cooperative approach for numerical optimization.

• A fast operator is proposed to capture the interdependencies among variables.

• Problem decomposition is performed based on the interdependencies.

• A cross-cluster mutation is proposed to optimize the subproblems.

• This approach is effective for large scale optimization problems.

Abstract

Cooperative optimization algorithms have been applied with success to solve many optimization problems. However, many of them often lose their effectiveness and advantages when solving large scale and complex problems, e.g., those with interacted variables. A key issue involved in cooperative optimization is the task of problem decomposition. In this paper, a fast search operator is proposed to capture the interdependencies among variables. Problem decomposition is performed based on the obtained interdependencies. Another key issue involved is the optimization of the subproblems. A cross-cluster mutation strategy is proposed to further enhance exploitation and exploration. More specifically, each operator is identified as exploitation-biased or exploration-biased. The population is divided into several clusters. For the individuals within each cluster, the exploitation-biased operators are applied. For the individuals among different clusters, the exploration-biased operators are applied. The proposed operators are incorporated into the original differential evolution algorithm. The experiments were carried out on CEC2008, CEC2010, and CEC2013 benchmarks. For comparison, six algorithms that yield top ranked results in CEC competition are selected. The comparison results demonstrated that the proposed algorithm is robust and comprehensive for large scale optimization problems.

Introduction

Optimization problems are rife in diverse fields such as mechanical engineering, compressed sensing, natural language processing, structure control, and bio-computing [1], [2], [3], [4], [5]. Researchers have to determine a set of model parameters or state-variables that provide the minimum or maximum value of a predefined cost or objective [6]. With the coming of internet of things (IoT) [7], many optimization problems are becoming more difficult, i.e., the problems are characterized by more variables with complicated interactions. Research on optimization problems has attracted the attention of researchers and many algorithms have been proposed. Though the existing optimizers have shown to be successful in solving moderate scale problems, many of them still suffer from the “curse of dimensionality”, which means that their performance will deteriorate as the dimensionality of the problem increases [8], [9]. Thus effective and efficient algorithms for large scale optimization have become essential requirements. In this paper, we aim at solving the large scale optimization problems and providing tools for scientists and engineers when they are solving real world problems from the involved disciplines.

Generally, the natural way to address the “curse of dimensionality” is to apply cooperative optimization, which can be regarded as an automatic approach to implement the divide-and-conquer strategy. A typical cooperative optimization algorithm can be summarized as follows [10]:

• Problem decomposition: decompose a large scale problem into smaller scale subproblems.

• Subproblem optimization: optimize each subproblem by means of a separate optimizer.

• Cooperative coordination: combine the subsolutions to obtain an entire solution.

A key issue with regards to the cooperative optimization is the task of problem decomposition. An appropriate decomposition algorithm should group interacted variables together so that the interdependencies among different subproblems are minimized. Based on whether the variable interdependencies are considered or not, the decomposition algorithms can be classified into two categories. Generally, the algorithms performed without considering variable interdependencies are simple and effective for separable problems, but have difficulty in solving nonseparable problems [10], [11], [12], [13], [14], [15], [16]. On the other hand, the algorithms performed by considering the variable interdependencies provide opportunities to solve large scale nonseparable problems [17], [18], [19], [20], [21], [22], [23], [24]. However, many of them either add extra computational burden to the algorithm or lack extensive variable interdependence learning.

Another key issue with regards to the cooperative optimization is the optimization of the subproblems. The widely used optimizers are inspired by nature phenomena, which include genetic algorithm (GA) [25], evolution programming (EP) [26], [27], evolution strategy (ES) [28], [29], differential evolution (DE) [6], [30], ant colony optimization (ACO) [31], particle swarm optimization (PSO) [32], [33], [34], [35], [36], [37], bacterial foraging optimization (BFO) [38], simulated annealing (SA) [39], tabu search (TS) [40], harmony search (HS) [35], [36], [40], etc. These optimizers facilitated research into the optimization of the subproblems. However, many of them are still not free from premature convergence for the complex multi-modal, rugged, and nonseparable problems.

As can be seen from the aforementioned, as far as cooperative optimization algorithms concerned, there still exists a big room to improve their performance through deeper studies. In this paper, we propose a variant of cooperative optimization algorithm. The study concentrates on the aforementioned two issues. To solve the task of problem decomposition, we propose a fast variable interdependence searching operator, which operates by recursively partitioning decision variables into blocks and identifying the interdependences among different blocks. Then we decompose a large scale problem into small scale subproblems based on the obtained interdependencies. To solve the task of subproblem optimization, we propose a cross-cluster mutation strategy, in which each operator is identified as exploration-biased or exploitation-biased. The population is divided into several clusters. For the individuals among different clusters, the exploration-biased operators are applied, and for the individuals within each cluster, the exploitation-biased operators are applied. By favoring search in the vicinity of each cluster and in the regions among different clusters, this strategy promotes efficient exploration as well as efficient exploitation. We further incorporate the proposed strategies into the original differential algorithm to perform optimization. The reason that we adopt differential evolution as base optimizer is that it has been frequently adopted and the resulting variants have been achieving top ranks in various competitions [30].

The reminder of this paper is organized as follows. Section 2 reviews the related works with regard to cooperative optimization and differential evolution. Section 3 gives the description of the proposed algorithm, which includes the fast variable interdependence learning method, and the cross-cluster mutation strategy. Section 4 presents the experimental results, followed by concluding remarks in Section 5.