Elsevier

Applied Soft Computing

Volume 13, Issue 9, September 2013, Pages 3853-3863
Applied Soft Computing

Free Pattern Search for global optimization

https://doi.org/10.1016/j.asoc.2013.05.004Get rights and content

Highlights

  • We propose a new algorithm called Free Pattern Search.

  • Some solutions are better than the solutions of some other approaches.

  • The convergence speed is much higher than some other approaches.

  • This new algorithm is scalability to the increasing of dimension.

Abstract

An efficient algorithm named Pattern search (PS) has been used widely in various scientific and engineering fields. However, even though the global convergence of PS has been proved, it does not perform well on more complex and higher dimension problems nowadays. In order to improve the efficiency of PS and obtain a more powerful algorithm for global optimization, a new algorithm named Free Pattern Search (FPS) based on PS and Free Search (FS) is proposed in this paper. FPS inherits the global search from FS and the local search from PS. Two operators have been designed for accelerating the convergence speed and keeping the diversity of population. The acceleration operator inspired by FS uses a self-regular management to classify the population into two groups and accelerates all individuals in the first group, while the throw operator is designed to avoid the reduplicative search of population and keep the diversity. In order to verify the performance of FPS, two famous benchmark instances are conducted for the comparisons between FPS with Particle Swarm Optimization (PSO) variants and Differential Evolution (DE) variants. The results show that FPS obtains better solutions and achieves the higher convergence speed than other algorithms.

Introduction

Nowadays, the problems become more and more complicated in scientific research, engineering, financial and management field and so on, which creates a demand of more powerful optimizer to solve them within limited time and memory. Over the last decades, nature-inspired algorithms took the responsibility for that and dedicated themselves to the complex problems. Lots of intelligence algorithms had been proposed such as genetic algorithm [1], [2] (GA), particle swarm optimization [3] (PSO), differential evolution [4] (DE), human search [5] (HS) and artificial bee colony algorithm [6] (ABC). They show a great potential for global optimization and have been applied to solve various engineering problems [7], [8] for their simplification and effectiveness [9].

However, almost all of the algorithms suffer from the prematurity. In order to get a higher quality solution and better performance, hybrid algorithms are used for handling this, such as CPSO [10] (hybrid PSO and Cellular automata), DSSA [11] (hybrid SA and Direct search), DE/BBO [12] (hybrid DE and BBO), DTS (hybrid TS and Direct search) [13]. All of these hybrid algorithms have achieved the good performance. However, the real problems become more and more complex, and the benchmarks which are used to test the performance of algorithms also become more and more complex, at the same time the dimensions of benchmarks increase as well, which makes them more difficult to be solved. Even lots of good algorithms have been designed, this road never ends. This research focuses on this topic and a new algorithm called Free Pattern Search (FPS) has been designed for optimization.

FPS is inspired by Pattern Search (PS) and Free Search (FS), and it extends PS into a population based formation. PS was firstly proposed by Hooke and Jeeves [14] in 1961. Torczon [15] provided a detailed formal definition of PS. As a kind of direct search, PS does not require gradient information at all, so it shares some similarities with modern evolutionary algorithms. The search step size of PS is very crucial for global convergence of PS and it should be reduced only when no increase or decrease in any one parameter further improved the fit [16], even though PS has been proofed to be a global convergence algorithm [15].

Traditional PS did not perform well on the complex and high dimension problems. Hvattum and Glover [17] proposed a new direct search method called SS (Scatter Search) and proved that HJPS (Hooke and Jeeves Pattern Search) was worse than CS (Compass Search) and SS even though the convergence properties of them were similar. Lots of researchers have been dedicated to modify PS for better performances [18], [19], [20]. In fact, the empirical convergence is different from theoretical convergence. The former one is the real performance, which has more impacts on the quality of the results.

FS [21], [22] is the other resource for FPS. Because of the self-regular management, FS which is a population based algorithm has good global search ability [22]. The acceleration part in FPS uses a FS-inspired part, exactly, a self-regular part to classify the population, and then PS take the responsibility to accelerate them. In order to enhance the exploitation phase and keep the diversity of population, FS works with PS to ensure the exploitation and exploration of FPS as well as other operators.

The rest of the paper is organized as follows. The next section introduces the traditional HJPS. The detailed description of FPS is presented in Section 3. In Section 4, two sets of experiment are applied to evaluate the performance of FPS. Section 5 is the conclusion and future researches.

Section snippets

Traditional Pattern Search

The traditional HJPS is a single-point search method, which generates a sequence of non-increasing solutions in whole iterations. The “pattern” which is very important is the neighborhood structure of the base point, and the points on the neighborhood structure are called trials. Fig. 1 shows the HJPS pattern on 2D. The white points are the trials of the black one (base point), as it can be seen clearly that the trials form a grid and the grid is the neighborhood structure of the base point

Free Pattern Search

This section presents a new algorithm inspired by HJPS and FS called Free Pattern Search. This algorithm integrates HJPS method to be the local search and some operators from FS to keep the diversity by avoiding the individuals’ repetitious search.

Experimental setup and results

To evaluate the performance of FPS, various famous benchmark instances have been used and two famous algorithms have been adopted to compare with the proposed FPS. The first one is the PSO variants, and the other one is the DE variants.

There are two sets of testing instances. The first set is conducted for the comparison with PSO variants using the traditional benchmarks, while DE variants are selected for the second set using the first ten functions in CEC 2005 benchmark set [26]. Two lastest

Discussion

PS is an efficient algorithm, but it hardly to solve more complex problems. FPS extends the traditional PS into a population-based formation, and achieves the good results. The interaction between the individuals is very important, and it can enhance the search ability. The performance of FPS is valid through the comparisons with PSO variants and DE variants. And the advantages of FPS are summarized in Table 11:

  • FPS promotes the famous HJPS and it ensures the enough local search. PS is an old

Conclusion and future researches

This research presents a new algorithm called FPS. The motivation behind FPS is to extend PS into a population-base algorithm and get a better performance including higher quality solutions and faster convergence speed. With the help of Free Search, acceleration operator is designed and adopted. It works together with throw operator to keep the diversity of the population and accelerate the searching speed if possible. Two sets of experiments have been carried out. And the comparisons between

Acknowledgements

This research work is supported by the Natural Science Foundation of China (NSFC) under Grant nos. 60973086 and 51005088, the National Basic Research Program of China (973 Program) under Grant no. 2011CB706804.

References (33)

  • K. Penev et al.

    Free Search—a comparative analysis

    Information Science

    (2005)
  • S.L. Sabat et al.

    Integrated learning particle swarm optimizer for global optimization

    Applied Soft Computing

    (2011)
  • Z.H. Cai et al.

    A clustering-based differential evolution for global optimization

    Applied Soft Computing

    (2011)
  • R. Salomon

    Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions

    Biosystems

    (1996)
  • Y.W. Leung et al.

    An orthogonal genetic algorithm with quantization for global numerical optimization

    IEEE Transactions on Evolutionary Computation

    (2001)
  • J. Kennedy et al.

    Particle swarm optimization

    Proceedings of IEEE International Conference on Neural Networks

    (1995)
  • Cited by (18)

    • Optimizing microstructure of a poroelastic layer with cylindrical pores for absorption properties

      2019, Mechanics Research Communications
      Citation Excerpt :

      The initial point for this method is given by the middle of the grid (ϕ, L). The optimisation is performed through iterations when the sequence of non-increasing function values is obtained [29]. The method is characterised by the mesh of step size Δ which is estimated in accordance with the constraints introduced in Eq. (25).

    • MPPT design of centralized thermoelectric generation system using adaptive compass search under non-uniform temperature distribution condition

      2019, Energy Conversion and Management
      Citation Excerpt :

      This paper aims to investigate the MPPT design of the centralized TEG system which can rapidly and reliably seek the GMPP under the effect of NTD condition, such that its generation efficiency can be maximized with the minimal implementation and maintenance costs of MPPT converter. CS is a classical pattern search method, which consists of three simple steps [32], e.g., (a) pattern matrix determination, (b) exploration move, and (c) pattern move. For a two-variable minimum optimization, it contains four exploration directions [31], i.e., east, west, south, and north, as briefly demonstrated in Fig. 4.

    • Quantification of aging mechanisms and inhomogeneity in cycled lithium-ion cells by differential voltage analysis

      2019, Journal of Energy Storage
      Citation Excerpt :

      A global optimization routine that can sufficiently approximate the DVA curves and provides further information about the dominating loss mechanism is needed. For this purpose, an optimization routine has been developed using the idea of the pattern search as for example explained in [46]. The steps that are necessary to find the best DVA fit are listed in the flowchart in Fig. 5.

    • Capacity planning with technology replacement by stochastic dynamic programming

      2017, European Journal of Operational Research
      Citation Excerpt :

      Each point is defined as a vector solution, and a better point has a lower objective function value. PS is widely known as an efficient algorithm and has been adopted in variety of scientific and engineering fields (Wen, Gao, Li, & Zhang, 2013). PS usually consumes less CPU time than GA (Basak, Sanyal, Nath, & Goswami, 2013; Franchini & Galeati, 1997; Wikipedia, 2013).

    • Potential of vehicle-to-grid ancillary services considering the uncertainties in plug-in electric vehicle availability and service/localization limitations in distribution grids

      2016, Applied Energy
      Citation Excerpt :

      The latter using derivative free algorithms seem effective. In [46], it is shown that Free Pattern Search (FPS) algorithm is scalable to the dimension increasing and performs better compared to the other evolutionary algorithms. This algorithm employs the HJPS method as a local search algorithm and two operators from FS to guarantee the diversity of search in order to inherit the global search.

    View all citing articles on Scopus
    View full text