Computational investigation of simple memetic approaches for continuous global optimization

Highlights

• Introduction of three different variants of the Memetic Differential Evolution method (see [20]).

• Computational investigation of these variants and their relation with the funnel properties of the functions to be optimized.

• Comparison with the existing literature about methods based on multiple local searches (in particular, Monotonic Basin Hopping).

• Comparison with the existing literature about memetic approaches.

• Experimental analysis of the proposed approaches to get more insight about them.

Abstract

In Locatelli et al. (2014) [20] a memetic approach, called MDE (Memetic Differential Evolution), for the solution of continuous global optimization problems, has been introduced and proved to be quite efficient in spite of its simplicity. In this paper we computationally investigate some variants of MDE. The investigation reveals that the best tested variant of MDE outperforms the original MDE itself, but also that the best variant depends on some properties of the function to be optimized. In particular, a greedy variant of MDE turns out to perform very well over functions with a single-funnel landscape, while another variant, based on a diversity measure applied to the members of the population, works better over functions with a multi-funnel landscape. A hybrid approach is also proposed which combines both the previous variants in order to obtain an overall performance which is good over all functions.

Introduction

The task of globally minimizing a multimodal objective function f over some compact domain $X\subset {\mathbb{R}}^n$ is a very difficult one. Different approaches exist based on the dimension of the search space and on the objective function and feasible domain׳s properties. The approaches range from exact (usually branch-and-bound) ones, suitable for highly structured problems (e.g., with a quadratic objective function and a polyhedral feasible domain) when the dimension n is not too large (say, few hundreds of variables), to heuristic ones where only few function evaluations are performed, suitable for problems where a single function evaluation is a rather costly operation. While we refer to [19] for a thorough discussion about all possible different approaches, here we focus our attention on approaches which are suitable for problems where local searches are a relatively cheap task but at the same time the huge number of local minimizers rules out the simplest approach based on multiple local searches, namely Multistart, where local searches are performed from different points randomly generated within the feasible region. In the algorithms we are going to discuss the points observed at each iteration are always local minimizers. To be more precise, the observed points are always the output of local search procedures, which are usually guaranteed to be stationary points but, in fact, are typically also local minimizers. In what follows we will always refer to these points as local minimizers, keeping in mind the clarification we have just made. Of course, observing local minimizers has a cost, since for each observation we need to perform a local search, which requires some function (and gradient) evaluations. However, this cost is often largely compensated. Indeed, when the landscape of the objective function is rough with high barriers between local minimizers, even points very close to the global minimizer may have large function values. If, as it is often the case, the function value is employed to evaluate the quality of a point and to decide whether to keep or to discard it, such points, though close to the global minimizer, will be discarded from further consideration. Local searches, by driving a point towards a local minimizer, remove the negative effect of high barriers between local minimizers.

In this paper we discuss three variants of the Memetic Differential Evolution (MDE) approach introduced in [20]. In that paper it was shown that MDE is very competitive, in terms of local searches needed to reach the global minimizer, with respect to other methods based on multiple local searches and, in particular, with respect to the Monotonic Basin Hopping method which will be discussed in Section 2.1. The primary aim of the paper is to show, through an extensive computational investigation, that each proposed variant improves, again in terms of local searches needed to reach the global minimizer, the original MDE approach under some given funnel properties of the objective function to be minimized. More precisely, the greedy variant (G-MDE) improves the performance of MDE when applied to single-funnel functions (but with some local search procedures also when applied to multi-funnel functions); the distance variant (D-MDE), by preserving a higher degree of diversity within the population, improves upon the original MDE approach over multi-funnel functions; finally, the hybrid approach (H-MDE), where the greedy and distance strategies are mixed, improves upon the original MDE over all the tested functions. We remark that none of the proposed approaches dominates all the others but this is consistent with the “no free lunch” theorem [43]. As a secondary aim we would like to show that, in spite of its extreme simplicity, one of the proposed variants, namely G-MDE, is competitive with some state-of-the art evolutionary approaches in terms of the quality of the solutions returned within a prefixed budget of function evaluations. The paper is structured as follows. In Section 2 we introduce a general scheme of a global optimization approach based on local searches and we briefly discuss some existing approaches which fit into this scheme, namely Multistart and Mononotic Basin Hopping (Section 2.1), memetic algorithms (Section 2.2), and MDE (Section 2.3) . In Section 3 we propose the three previously mentioned simple variants of MDE. In Section 4 we present the set of test problems on which we compare the different variants and we discuss the results of the computational experiments. We also perform an experimental analysis of the proposed approaches in order to better understand their behavior. Finally, in Section 5 we draw some conclusions and we discuss some possible future developments.