Elsevier

Applied Soft Computing

Volume 13, Issue 3, March 2013, Pages 1464-1477
Applied Soft Computing

A comparative study of different local search application strategies in hybrid metaheuristics

https://doi.org/10.1016/j.asoc.2012.05.016Get rights and content

Abstract

This paper presents results of a comparative study with the objective to identify the most effective and efficient way of applying a local search method embedded in a hybrid algorithm. The hybrid metaheuristic employed in this study is called “DE–HS–HJ” because it is comprised of two cooperative metaheusitic algorithms, i.e., differential evolution (DE) and harmony search (HS), and one local search (LS) method, i.e., Hooke and Jeeves (HJ) direct search. Eighteen different ways of using HJ local search were implemented and all of them were evaluated with 19 problems, in terms of six performance indices, covering both accuracy and efficiency. Statistic analyses were conducted accordingly to determine the significance in performance differences. The test results show that overall the best three LS application strategies are applying local search to every generated solution with a specified probability and also to each newly updated solution (NUS + ESP), applying local search to every generated solution with a specified probability (ESP), and applying local search to every generated solution with probability and also to the updated current global best solution (EUGbest + ESP). ESP is found to be the best local search application strategy in terms of success rate. Integrating it with NUS further improve the overall performance. EUGbest + ESP is the most efficient and it is also able to achieve high level of accuracy (the fourth place in terms of success rate with an average above 0.9).

Highlights

► This paper is the first comprehensive comparative study of local search application strategies in hybrid metaheuristic algorithms. ► The hybrid metaheuristic algorithm employed in the study is state-of-the-art, composed of two cooperative metaheuristic algorithms (differential evolution and harmony search) enhanced by the Hooke and Jeeves local search. ► The top three best out of 18 different strategies are identified based on the results obtained in testing a suite of 19 problems. The results are expected to be useful for deciding how to design a hybrid metaheuristic algorithm.

Introduction

The term meta-heuristic, first introduced by Glover [1], generally refers to any optimization methods that implements certain strategies for searching the space to find the global optima. Although there is no theoretical guarantee that they can always find the global optima, it has been reported that meta-heuristic algorithms are able to produce very good results, as reported in research and in practice. Furthermore, meta-heuristics are known to have the advantage to deal with optimization problems with non-convex and non-differentiable functions, and problems difficult to be explicitly modeled.

In the optimization literature, there are generally two types of optimizing methods: global and local. Global optimizing methods (GOM) are designed to explore the search space and typically use a lot of time to find an optimal/near-optimal solution if used alone. On the other hand, local optimizing methods (LOMs) are designed to exploit a local area, but they are usually not good at exploring wide area and hence not applied alone for global optimization. Roughly speaking, exploration aims at locating promising regions within the search domain and exploitation zeros in on the optimum in the detected promising regions. Due to their respective strength and weakness, there is motivation for the hybridization of GOMs and LOMs. The key to achieving high performance for any meta-heuristic algorithm is to maintain a good balance between exploitation and exploration during search. To this end, there has been much interest in hybrid metaheuristics.

Hybrid metaheuristics are algorithms that do not purely follow the concept of one single traditional metaheuristic, rather they combine various algorithmic ideas, sometimes from outside of the metaheuristic field. The hybridizations of different algorithmic concepts is usually motivated by the desire to obtain better performing systems that exploit and unite advantages of the individual pure strategies, i.e. such hybrids are believed to benefit from synergy. Talbi [2] develops taxonomy of hybrid metaheuristics in an attempt to provide a common terminology and classification mechanisms. By combing various view points, Raidl [3] groups hybrids of metaheuristics according to several criteria, which include algorithms used, level of hybridization, order of execution, and control strategy. Numerous hybrid metaheuristics have been developed and applied to both combinatorial and continuous optimization problems.

A very common way of designing hybrid metaheuristic is based on a diversification phase followed by intensification. Both the diversification and intensification phases could be computationally expensive. To decrease the computational cost, it is desirable to stop the diversification phase earlier in order to start the intensification phase sooner. However, switching to intensification too early increases the possibility of being trapped in local minima. Determining the optimal switching time between the diversification phase and the intensification phase is thus very important for any hybrid meta-heuristic algorithm. To the best of our knowledge no study has been carried out to date to systematically evaluate and compare the entire spectrum of different strategies for applying a local search (LS) method.

To fill in this gap, this study aims to carry out such a comparative study of 18 local search application strategies incorporated into a selected hybrid metaheuristic. All different local search strategies are tested on 19 engineering design problems. Results obtained in this study are compared based on six performance indices, covering both accuracy and efficiency. The objective is to identify the most effective and efficient way of applying a local search method embedded in a hybrid algorithm. The hybrid meta-heuristic algorithm employed in this study is one that is recently developed, as a further improvement of Liao [4]. The basic building block of the hybrid includes two cooperative meta-heuristic algorithms, i.e., differential evolution (DE) and harmony search (HS), and a local search method, i.e. Hooke and Jeeves direct search (HJ). The reason for using two cooperative metaheuristic algorithms is because they outperform those using one alone (see Section 2 for more details). Population-based metaheuristics such as differential evolution (DE) and harmony search (HS) are stochastic in nature. They are very good at diversification and identifying promising searching areas (in order to jump out from those local optima). The combined use of different search mechanism further enhances the robustness of global search with respect to changes in the problem instances. Similar observations were reported in other studies (Crainic et al. [5], Pelta et al. [6], and Cadenas et al. [7]).

The HJ local searcher, on the other hand, is a highly efficient deterministic direct search method superior in intensification. It enhances the hybrid's ability to further exploit into those promising areas by a “direct search”. The phase “direct search” by Hooke and Jeeves [8] is used to describe sequential examination of trial solutions involving comparison of each trial solution with the “best” obtained up to that time together with a strategy for determining (based on the objective function) what the next trial solution will be. It uses the steepest ascent pivot rule in which it performs a coordinate search for each dimension of the trial solution so that it deeply exploits all the neighborhood of the trial solution so that the nearest local optimum within that area is guaranteed. Therefore, the new hybrid achieves a better balance between diversification and intensification which will enhance its ability to find the optimum more efficiently. Nevertheless, the issue of local search application is particularly important when the local-search method is of high computational cost, as the Hooke and Jeeves used here.

In light of the No Free Lunch (NFL) theorem for optimization (Wolpert and Macready [9]), it is important to emphasize that the focus of this paper is on constrained mixed integer engineering design problems. The design of a product, process, or system can be affected by a large number of factors interacting in complicated ways. Engineering design optimization problems normally have mixed (e.g., continuous and discrete) design variables, nonlinear objective functions and nonlinear constraints. We are interested in knowing whether there exists an optimal local search application strategy that is able to solve a wide range of engineering design problems. The results obtained by this study do reflect the behavior of this limited range of problems being considered and one should not generalize those results to problems from other domains without double checking.

The rest of this paper is organized as follows: Section 2 gives the literature review. Section 3 briefly introduces the components of the hybrid meta-heuristic algorithm employed. Section 4 describes 18 different strategies of applying local search. Section 5 presents the test results, followed by the discussion in Section 6. Finally, the paper is concluded in Section 7.

Section snippets

Literature review

This section reviews existing hybrid meta-heuristic algorithms for constrained mixed-integer optimization, paying special attention to the local search method used and how it was used. Summarizing how local search is used, a total of 11 different strategies can be distinguished, as detailed in the first column of Table 1 (Refs. [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37],

The DE–HS–HJ hybrid

In this section, the hybrid metaheuristic algorithm used in our study and its major components are described. Fig. 1 depicts the overall flowchart of the “DE–HS–HJ” metaheuristic. Because of space limitation, it is not possible to draw all 18 local search application strategies investigated. Moreover, including all of them into one flowchart will be very confusing and messy. Hence, only two strategies are presented, i.e., applying to every solution with probability (LS 6 in Section 4) and

Different local search application strategies

In order to investigate the effect of different local search application strategies, the HJ local search method is chosen because it performs better than random walk with direction exploitation (RWDE) used in Liao [4]. The HJ local search can be integrated with the MDE’–IHS cooperative meta-heuristic in many different ways. The criterion used to stop a run is either achieving the global optimum (or the known best solution) within 10−6 error or exceeding the pre-specified maximum number of

Test problems and results

The performances of all the local search application strategies as described in Section 4 were verified experimentally using a set of 19 constrained mixed integer optimization problems. All programs were coded in Matlab and all executions were made on a HP Pavilion a4317c with AMD Athlon™ II × 2@ 2.70 GHz.

Discussion

The test results presented in Section 5.3 provide us an overall picture of the performance of different LS application strategies. However, it does not show us detailed information about how each strategy performs on each function individually. To this end, the following tables are prepared for further discussion. Table 5 gives the success rate of each strategy in solving each problem. The results indicate that Problems 13 and 15 are the most difficult two among all. Problem 13 has an average

Conclusions

This paper has presented a comprehensive comparative study of 18 local search application strategies incorporated into a selected hybrid metaheuristic. The hybrid metaheuristic algorithm employed in this study is an improvement over a recent published work. All local search application strategies were tested with 18 benchmark functions plus one additional supply chain consignment problem using the same set of algorithmic parameters for fair comparison. According to the test results, ESP is the

References (57)

  • S. Salhi et al.

    A hybrid algorithm for identifying global and local minima when optimizing functions with many minima

    European Journal of Operational Research

    (2004)
  • M.H. Mashinchi et al.

    Hybrid optimization with improved tabu search

    Applied Soft Computing

    (2011)
  • L. Wei et al.

    A niche hybrid genetic algorithm for global optimization of continuous multimodal functions

    Applied Mathematics and Computation

    (2005)
  • S.K.S. Fan et al.

    A genetic algorithm and a particle swarm optimizer hybridized with a Nelder–Mead simplex search

    Computers & Industrial Engineering

    (2006)
  • K. Miettinen et al.

    Efficient hybrid methods for global continuous optimization based on simulated annealing

    Computers & Operations Research

    (2006)
  • P. Vasant et al.

    Hybrid genetic algorithms and line search method for industrial production planning with non-linear fitness function

    Engineering Applications of Artificial Intelligence

    (2009)
  • P. Vasant et al.

    Hybrid pattern search and simulated annealing for fuzzy production planning problems

    Computers and Mathematics with Applications

    (2010)
  • A.-R. Hedar et al.

    Tabu search directed by direct search methods for nonlinear global optimization

    European Journal of Operational Research

    (2006)
  • M. Fesanghary et al.

    Hybridizing harmony search algorithm with sequential quadratic programming for engineering optimization problems

    Computer Methods in Applied Mechanics and Engineering

    (2008)
  • N. Baskar et al.

    Selection of optimal machining parameters for multi-tool milling operations using a memetic algorithm

    Journal of Materials Processing Technology

    (2006)
  • H. Schmidt et al.

    A combined heuristic optimization technique

    Advances in Engineering Software

    (2005)
  • S.F. Hwang et al.

    Improving real-parameter genetic algorithm with simulated annealing for engineering problems

    Advances in Engineering Software

    (2006)
  • R. Angira et al.

    Optimization of process synthesis and design problems: a modified differential evolution approach

    Chemical Engineering Science

    (2006)
  • K. Deb

    An efficient constraint handling method for genetic algorithms

    Computer Methods in Applied Mechanics and Engineering

    (2000)
  • M. Mahdavi et al.

    An improved harmony search algorithm for solving optimization problems

    Applied Mathematics and Computation

    (2007)
  • J.C. Bezdek et al.

    FCM: the fuzzy c-means clustering algorithm

    Computers and Geosciences

    (1984)
  • T. Yokota et al.

    Genetic algorithm for non-linear mixed integer programming problems and its application

    Computers and Industrial Engineering

    (1996)
  • T.C. Chen

    IAs based approach for reliability redundancy allocation problems

    Applied Mathematics and Computation

    (2006)
  • Cited by (30)

    • Metaheuristic-based inverse design of materials – A survey

      2020, Journal of Materiomics
      Citation Excerpt :

      The comparative results are expected to shed light on developing new effective local search operators. Furthermore, how local search should be best applied is also a relevant issue, as shown in Ref. [114]. Therefore, developing new effective local search operators that will work well for inverse design of materials together with how it is applied is a possible future research direction.

    View all citing articles on Scopus
    View full text