GGA: A modified genetic algorithm with gradient-based local search for solving constrained optimization problems

Abstract

In the last few decades, genetic algorithms (GAs) demonstrated to be an effective approach for solving real-world optimization problems. However, it is known that, in presence of a huge solution space and many local optima, GAs cannot guarantee the achievement of global optimality. In this work, in order to make GAs more effective in finding the global optimal solution, we propose a hybrid GA which combines the classical genetic mechanisms with the gradient-descent (GD) technique for local searching and constraints management. The basic idea is to exploit the GD capability in finding local optima to refine search space exploration and to place individuals in areas that are more favorable for achieving convergence. This confers to GAs the capability of escaping from the discovered local optima, by progressively moving towards the global solution. Experimental results on a set of test problems from well-known benchmarks showed that our proposal is competitive with other more complex and notable approaches, in terms of solution precision as well as reduced number of individuals and generations.

Introduction

Optimization has received an ever-growing interest in recent years and many novel optimization approaches and techniques are continuously developed and used to solve real-world problems. It essentially aims at finding the best solution from a huge constrained solution space (search space). More in general, it deals with the study of decision-making problems in which one or more objective functions need to be minimized or maximized simultaneously under given constraints. In real life we incur in many examples of optimization problems (OPs), involving several fields of science, such as engineering, economics, manufacturing, marketing, finance, transportation, communication networks, and more. Problems in these fields are characterized by having restrictions coming from technical limitations, business plans, human needs, laws, and so forth. Therefore, a solution of an OP should be able to choose the decision alternative that considers all the constraints, while optimizing the involved objectives. Two main optimization options exist, respectively referred to as exact and heuristic methods. The former guarantees to find the best solution to the problem (absolute optimum), while the latter tries to achieve a satisfactory near-optimal result with a significantly reduced computational effort. Clearly, the exact method is desirable, but when the problem dimensions grow, the computational complexity of the adopted solution technique (usually measured in terms of execution time and memory space) becomes an essential concern. In these cases, heuristic approaches are used, and, even though they are not able to guarantee the convergence to an optimal solution, they result in good performances. Moreover, heuristic approaches are also used when the exact ones fail in finding the solution. Evolutionary algorithms (EAs) are popular examples of heuristics. One of the key aspects of EAs is that they are multi-point search-based, and hence are intrinsically parallel. This allows them to explore the solution space in multiple directions at once, and hence to have a vision progressively more and more global of the search space. Today, they are successfully used for problem-solving in many fields, and in the last few decades GAs, differential evolution (DE), and particle swarm optimization (PSO) demonstrated their effectiveness for solving many complex real-world optimization problems. Most of these problems are characterized by a huge solution space with many local optima. In such a situation, EAs may be entrapped in some local optima, and cannot reach global optimality (premature convergence problem). Especially for GAs, this may occur if the search process requires many generations, which may lead to low diversity in the population if the exploration step is not properly designed. Indeed, as the number of iteration increases, the number of individuals approaching the optimum increases. Thus, the crossover operator is forced to choose similar individuals. Furthermore, since EAs are a non-deterministic class of algorithms, the solution may vary at each new run, depending also on the initial population. Possible approaches to address this issue include running EA many times, and/or increasing the number of individuals in the involved population at the expense of time and space efforts. To increase the chances of finding out an optimal solution, a combination of local search algorithms with different optimization approaches may be used, which takes advantage of each approach’s strengths and minimizes their weaknesses. They are known as hybrid methods, and when the combination involves EAs they are also called memetic algorithms [31]. The presence of a large number of approaches that can be used to solve OPs puts researchers faced with the problem of choosing the best EA-based algorithm. It should be noted that even though DE achieves good performance in searching global optimum, it is slow and the parameters are problem-dependent [33]. Besides, it is not completely free from the premature convergence and stagnation problems, and its performance decreases with increasing dimensionality of the problem. Many notable solutions have been proposed to improve the basic DE approach. Such solutions are often complex, which is not necessarily the best way to design a good algorithm. Rather, the implementation of an easy local search algorithm, if properly designed, is often the best option to obtain a performance that outperforms more complex approaches [12]. Another important way of looking into Memetic design is by fitness landscape analysis [18]. However, this approach relies on studying the nature of the problem before deciding on the right approach to be used.

The main aim of our work is to provide a solution as independent as possible from the nature of the problem, while ensuring solution precision as well as reduced number of individuals and generations. GA-based algorithms seem to be the best candidates for achieving these results. Indeed, their notable strength of not needing any prior knowledge about the problem to be solved makes them very attractive. Also, they implement all the basic genetic operators, such as selection, crossover, and mutation, ensuring a complete and granular control of the effects of the GA-based designed solutions in solving OPs. On the other hand, as it is known, PSO is significantly different from GA, because it does not provide genetic operators like crossover and mutation.

In order to make GAs more effective and efficient in finding solutions for constrained and unconstrained Single Objective Optimization Problems (SOOPs), we propose a hybrid approach that combines GAs with two gradient descent (GD)-based algorithms and uses variants of both crossover and mutation GA operators. The resulting solution is an algorithm called GGA (Gradient-based Genetic Algorithm) able to confer to GAs the capability of finding the optimal solution through a reduced number of generations and fewer individuals. The basic idea is to exploit the capabilities of GD for refining local solutions and use them as more favorable starting points in the GA. As a result, the GA gets the ability to escape from the discovered local optima, and progressively moves towards the global solution. The GA is used to diversify the search in order to find promising new search areas (exploration), while a GD-based algorithm (named GDC – Gradient Descent with Constraints) is employed to examine more deeply a given area (exploitation). This guarantees a balancing between exploitation and exploration of the given search space. To deal with constraints, an algorithm called RGDA (Resultant Gradient Descent Algorithm) is provided as alternative to popular solutions, such as the penalty-based approach [47] or the augmented Lagrangian method [8]. Also, the effects of the proposals are emphasized by the usage of two provided crossover and mutation GA operators, named SPC (Sliding Point Crossover) and BGM (Bounded Genes Mutation), respectively. Fig. 1 depicts a block diagram of the proposed approach.

The remainder of the paper is organized as follows. Section 2 presents the related works. Preliminaries are shown in Section 3 whereas in Section 4 the proposed algorithms are described in detail. Experimental results and their comparison with other approaches available in literature are the subject of Section 5. Finally, conclusions and future research issues are reported in Section 6.