Influence of randomization strategies and problem characteristics on the performance of Differential Search algorithm

Highlights

• The performance of Differential Search in different conditions was studied.

• The influence of random number generators and function evaluation was also considered.

• The case studies included 9 CEC2013 Benchmarks and 2 chemical engineering problems.

• Differential Search with Weibull distribution had the best results.

Abstract

In this work, the influence of different random number generators, problem dimensionality, and number of function evaluations on the optimization efficiency of Differential Search algorithm is presented in detail. Two types of random number generators were taken into account: discrete and continuous. Different combinations between the dimensionality property and the number of function evaluation setting were tested on: i) a set of benchmark functions from the CEC 2013 special session on real parameter optimization; and ii) 2 chemical engineering problems (optimal operation of an alkylation unit and heat exchanger network design). Also, a comparison with other optimizers was performed. It was found that, in similar conditions, the performance of the algorithm (in terms of the best solutions) varies substantially depending on the distribution used. In case of the benchmark problems, the best solutions were obtained for Binomial and Weibull distribution. For the separable functions is was observed that, indifferent of the distribution used, the algorithm was not able to find acceptable solution within the constraint represented by the number of function evaluations. In the case of the alkylation problem the best solutions were obtained by the Weibull distribution, the performance of the Differential Search algorithm being comparable to other optimizers such as Differential Evolution. In the case of the heat exchanger, three different distribution provided near optimal solutions (Binomial, ChiSquare and Weibull).

Introduction

Optimization is an important aspect of all natural processes, with a tendency to consume the minimum amount of resources and to obtain the highest gain. This is also applicable to research studies, where in order to solve specific problems, the best optimizers are employed. Although an optimization state is always desirable and there is a proliferation of optimization methods, there is no universal approach for solving all the optimization problems [1]. The no-free lunch theorem states that one cannot expect a single algorithm to outperform all others in all possible problem instances and therefore researchers focus on developing new algorithms that can be efficiently applied to large areas of problems [2], [3], [4].

The inspiration for these methods varies, in the latest years, nature (biological systems) ranking on top. The field study associated to this methods is called Bio-Inspired Computing, the areas of research including Evolutionary Algorithms (EAs) – inspired from evolution, Swarm Optimization – inspired from the swarm behavior, Artificial Neural Networks – inspired from mechanisms of the mammalian brain, Artificial Immune Systems – inspired from the response of the vertebrate immune system. As it can be observed, the list is quite impressive and the multitude of algorithms proposed is rising. As each algorithm has its specific characteristics that are either inherited from the inspiration source or are consequences of the internal methods applied, an in-depth performance analysis of each approach is required in order to identify the areas where its application is most suited. The majority of these algorithms are global optimizers, a global optimization procedure implying finding the best set of parameters that optimize an objective function [5].

In this work, a relatively new algorithm, called Differential Search (DS) [6] and inspired from the migration movement of organisms was studied in detail and its performance on different types of problems assessed. The scope was to determine the influence of characteristics of the problem being solved (dimensionality, linearity, single or multi-objective, existence of multiple minima) on the algorithm’s capability to find solutions in the close vicinity of the minima (considering that the problem is a minimization one).

As it was stated previously, DS is a relatively new algorithm and therefore the available literature and studies regarding its application and behavior are quite scarce. It was proposed in 2012 by Civicioglu [6] as a mean to transform the geocentric cartesian coordinates into geodetic coordinates. It simulates the Brownian like random walk movement used by migration organisms. The main idea of DS is that an artificial superorganisms migrates to the global minimum by testing if some randomly positions are temporarily suitable.

In an attempt to raise the performance, Gan and Duan [7] proposed a chaotic based DS in which the property of a chaotic variable is integrated in order to improve the search capabilities. This variant, called CDS is applied for image extraction and image enhancement and, in order to further improve its performance, a Lateral Inhibition (LI) mechanism is employed for image pre-processing, resulting in a new algorithm called CDS-LI. A comparison with the classical DS, CDS and PSO showed that for the problem at hand, the best results are obtained with the CDS-LI version. Another work in which DS is applied to solve a specific problem is the one of Goswami and Chakraborty [8] where parametric optimization of three electrochemical micromachining systems is performed by the classical variant of DS proposed in [6]. Similar to the other studies, results pointed out that DS is an acceptable global optimization tool. The capabilities of DS were also tested on a set of three antenna array synthesis problems: i) placing wide nulls on the array pattern by controlling the amplitude; ii) obtaining array patterns with individual nulls imposed at the interference directions by controlling the amplitude-only, phase-only and position-only; iii) failure correction [9]. As in the previous cases, the results were in the accepted interval, which pointed out DS as a good alternative to other antenna array synthesis algorithms.

A raising trend of using DS to solve different problems was observed in the last two years, where the area of problems enlarged to include the chemical engineering domain, structure design and other engineering problems [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24]. However, its strengths and weaknesses influenced by the characteristics of the problem being solved (linearity, dimensionality) and by the internal approaches used (stop criteria, parameter control settings, initialization procedure) are not studied in detail or not studied at all. In order to determine its efficiency, Civicioglu [6] performed a series of tests on a set of benchmark functions with a dimensionality varying from 2 to 30. The statistical analysis employed was the Wilcoxon Rank Sum Test and the results showed that in the majority of cases DS outperforms Artificial Bee Colony (ABC), different variants of Differential Evolution (DE) algorithm or Particle Swarm Optimization (PSO). As no emphasis was put on the dimensionality of the benchmark functions, initialization procedures, or stop criteria, it is clear that an extensive study related to these aspects is required.

This lack of knowledge can influence the decision of an end-user to choose another approach, even if DS has the potential to provide faster and more accurate solutions. Consequently, in this work, the influence of different types of random distributions, dimensionality of the problem and number of iterations on the performance of the DS algorithm is tested for different benchmark and real-life problems from the chemical engineering area. The scope is to determine the characteristics of the problems for which DS performs better and the areas where improvements are required in order to raise performance.