Free Pattern Search for global optimization
Graphical abstract
Introduction
Nowadays, the problems become more and more complicated in scientific research, engineering, financial and management field and so on, which creates a demand of more powerful optimizer to solve them within limited time and memory. Over the last decades, nature-inspired algorithms took the responsibility for that and dedicated themselves to the complex problems. Lots of intelligence algorithms had been proposed such as genetic algorithm [1], [2] (GA), particle swarm optimization [3] (PSO), differential evolution [4] (DE), human search [5] (HS) and artificial bee colony algorithm [6] (ABC). They show a great potential for global optimization and have been applied to solve various engineering problems [7], [8] for their simplification and effectiveness [9].
However, almost all of the algorithms suffer from the prematurity. In order to get a higher quality solution and better performance, hybrid algorithms are used for handling this, such as CPSO [10] (hybrid PSO and Cellular automata), DSSA [11] (hybrid SA and Direct search), DE/BBO [12] (hybrid DE and BBO), DTS (hybrid TS and Direct search) [13]. All of these hybrid algorithms have achieved the good performance. However, the real problems become more and more complex, and the benchmarks which are used to test the performance of algorithms also become more and more complex, at the same time the dimensions of benchmarks increase as well, which makes them more difficult to be solved. Even lots of good algorithms have been designed, this road never ends. This research focuses on this topic and a new algorithm called Free Pattern Search (FPS) has been designed for optimization.
FPS is inspired by Pattern Search (PS) and Free Search (FS), and it extends PS into a population based formation. PS was firstly proposed by Hooke and Jeeves [14] in 1961. Torczon [15] provided a detailed formal definition of PS. As a kind of direct search, PS does not require gradient information at all, so it shares some similarities with modern evolutionary algorithms. The search step size of PS is very crucial for global convergence of PS and it should be reduced only when no increase or decrease in any one parameter further improved the fit [16], even though PS has been proofed to be a global convergence algorithm [15].
Traditional PS did not perform well on the complex and high dimension problems. Hvattum and Glover [17] proposed a new direct search method called SS (Scatter Search) and proved that HJPS (Hooke and Jeeves Pattern Search) was worse than CS (Compass Search) and SS even though the convergence properties of them were similar. Lots of researchers have been dedicated to modify PS for better performances [18], [19], [20]. In fact, the empirical convergence is different from theoretical convergence. The former one is the real performance, which has more impacts on the quality of the results.
FS [21], [22] is the other resource for FPS. Because of the self-regular management, FS which is a population based algorithm has good global search ability [22]. The acceleration part in FPS uses a FS-inspired part, exactly, a self-regular part to classify the population, and then PS take the responsibility to accelerate them. In order to enhance the exploitation phase and keep the diversity of population, FS works with PS to ensure the exploitation and exploration of FPS as well as other operators.
The rest of the paper is organized as follows. The next section introduces the traditional HJPS. The detailed description of FPS is presented in Section 3. In Section 4, two sets of experiment are applied to evaluate the performance of FPS. Section 5 is the conclusion and future researches.
Section snippets
Traditional Pattern Search
The traditional HJPS is a single-point search method, which generates a sequence of non-increasing solutions in whole iterations. The “pattern” which is very important is the neighborhood structure of the base point, and the points on the neighborhood structure are called trials. Fig. 1 shows the HJPS pattern on 2D. The white points are the trials of the black one (base point), as it can be seen clearly that the trials form a grid and the grid is the neighborhood structure of the base point
Free Pattern Search
This section presents a new algorithm inspired by HJPS and FS called Free Pattern Search. This algorithm integrates HJPS method to be the local search and some operators from FS to keep the diversity by avoiding the individuals’ repetitious search.
Experimental setup and results
To evaluate the performance of FPS, various famous benchmark instances have been used and two famous algorithms have been adopted to compare with the proposed FPS. The first one is the PSO variants, and the other one is the DE variants.
There are two sets of testing instances. The first set is conducted for the comparison with PSO variants using the traditional benchmarks, while DE variants are selected for the second set using the first ten functions in CEC 2005 benchmark set [26]. Two lastest
Discussion
PS is an efficient algorithm, but it hardly to solve more complex problems. FPS extends the traditional PS into a population-based formation, and achieves the good results. The interaction between the individuals is very important, and it can enhance the search ability. The performance of FPS is valid through the comparisons with PSO variants and DE variants. And the advantages of FPS are summarized in Table 11:
- •
FPS promotes the famous HJPS and it ensures the enough local search. PS is an old
Conclusion and future researches
This research presents a new algorithm called FPS. The motivation behind FPS is to extend PS into a population-base algorithm and get a better performance including higher quality solutions and faster convergence speed. With the help of Free Search, acceleration operator is designed and adopted. It works together with throw operator to keep the diversity of the population and accelerate the searching speed if possible. Two sets of experiments have been carried out. And the comparisons between
Acknowledgements
This research work is supported by the Natural Science Foundation of China (NSFC) under Grant nos. 60973086 and 51005088, the National Basic Research Program of China (973 Program) under Grant no. 2011CB706804.
References (33)
Genetic algorithms for real parameter optimization
- et al.
Human evolutionary model: a new approach to optimization
Information Science
(2007) - et al.
Parameter optimization of a multi-pass milling process using non-traditional optimization algorithms
Applied Soft Computing
(2010) - et al.
Optimization of multi-pass turning operations using ant colony system
International Journal of Machine Tools and Manufacture
(2003) - et al.
Cellular particle swarm optimization
Information Science
(2011) - et al.
Direct search methods: then and now
Journal of Computational and Applied Mathematics
(2000) - et al.
Finding local optima of high-dimensional functions using direct search methods
European Journal of Operational Research
(2009) - et al.
A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation
Journal of Computational Physics
(2010) - et al.
A heuristic iterated-subspace minimization method with pattern search for unconstrained optimization
Computers & Mathematics with Applications
(2009) Solving unconstrained optimization problem with a filter-based nonmonotone pattern search algorithm
Applied Mathematics and Computation
(2008)
Free Search—a comparative analysis
Information Science
Integrated learning particle swarm optimizer for global optimization
Applied Soft Computing
A clustering-based differential evolution for global optimization
Applied Soft Computing
Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions
Biosystems
An orthogonal genetic algorithm with quantization for global numerical optimization
IEEE Transactions on Evolutionary Computation
Particle swarm optimization
Proceedings of IEEE International Conference on Neural Networks
Cited by (18)
Competitive adsorption of methylene blue and Pb (II) ions on the nano-magnetic activated carbon and alumina
2020, Materials Chemistry and PhysicsOptimizing microstructure of a poroelastic layer with cylindrical pores for absorption properties
2019, Mechanics Research CommunicationsCitation Excerpt :The initial point for this method is given by the middle of the grid (ϕ, L). The optimisation is performed through iterations when the sequence of non-increasing function values is obtained [29]. The method is characterised by the mesh of step size Δ which is estimated in accordance with the constraints introduced in Eq. (25).
MPPT design of centralized thermoelectric generation system using adaptive compass search under non-uniform temperature distribution condition
2019, Energy Conversion and ManagementCitation Excerpt :This paper aims to investigate the MPPT design of the centralized TEG system which can rapidly and reliably seek the GMPP under the effect of NTD condition, such that its generation efficiency can be maximized with the minimal implementation and maintenance costs of MPPT converter. CS is a classical pattern search method, which consists of three simple steps [32], e.g., (a) pattern matrix determination, (b) exploration move, and (c) pattern move. For a two-variable minimum optimization, it contains four exploration directions [31], i.e., east, west, south, and north, as briefly demonstrated in Fig. 4.
Quantification of aging mechanisms and inhomogeneity in cycled lithium-ion cells by differential voltage analysis
2019, Journal of Energy StorageCitation Excerpt :A global optimization routine that can sufficiently approximate the DVA curves and provides further information about the dominating loss mechanism is needed. For this purpose, an optimization routine has been developed using the idea of the pattern search as for example explained in [46]. The steps that are necessary to find the best DVA fit are listed in the flowchart in Fig. 5.
Capacity planning with technology replacement by stochastic dynamic programming
2017, European Journal of Operational ResearchCitation Excerpt :Each point is defined as a vector solution, and a better point has a lower objective function value. PS is widely known as an efficient algorithm and has been adopted in variety of scientific and engineering fields (Wen, Gao, Li, & Zhang, 2013). PS usually consumes less CPU time than GA (Basak, Sanyal, Nath, & Goswami, 2013; Franchini & Galeati, 1997; Wikipedia, 2013).
Potential of vehicle-to-grid ancillary services considering the uncertainties in plug-in electric vehicle availability and service/localization limitations in distribution grids
2016, Applied EnergyCitation Excerpt :The latter using derivative free algorithms seem effective. In [46], it is shown that Free Pattern Search (FPS) algorithm is scalable to the dimension increasing and performs better compared to the other evolutionary algorithms. This algorithm employs the HJPS method as a local search algorithm and two operators from FS to guarantee the diversity of search in order to inherit the global search.