A multi-information fusion “triple variables with iteration” inertia weight PSO algorithm and its application
Introduction
With the in-depth understanding of the mechanism of biological evolution and optimization problems, many methods to solve complex optimization problems have been proposed, such as particle swarm optimization (PSO), genetic algorithm (GA), simulated annealing algorithm (SAA), ant colony optimization (ACO) and other meta-heuristic algorithms [1], [2], [3], [4], [5], [6], [7], [8], [9]. Meta-heuristic algorithm simulates the biological species and weather phenomena in nature as an optimization algorithm, which is more popular because of its fast searching speed and low computational complexity. Especially in recent years, grey wolf optimization (GWO) [4], [5], dragonfly algorithm (DA) [6], [7], polar bear optimization (PBO) [8], [9], etc., have the advantages of effective global exploration and accurate local exploitation. Compared with other optimization algorithms, PSO [10] has the advantages of simple principle, less adjustable parameters, easy implementation and better optimization performance. At present, PSO algorithm has been widely used in training of neural network, multi-objective optimization, fuzzy control system, robot path planning, image segmentation, feature selection and classifier parameter optimization [11], [12], [13], [14], [15], [16], [17].
Although particle swarm optimization (PSO) has many advantages, it also has some shortcomings, for example, it is easy to fall into local optimal solution and depends on parameters. Therefore, many scholars have improved it, including the improvement of inertia weight parameters, the improvement of the overall topology and the combination with other algorithms. Inertial weight is an important parameter in particle swarm optimization (PSO), because it can maintain the diversity of particles and effectively balance the global exploration and local exploitation of particles. Therefore, since the proposed PSO algorithm with inertial weight, many scholars have proposed many strategies to improve the inertial weight.
Inertial weight was first introduced into PSO by Shi and Eberhart in 1998 [18]. At that time, inertial weight was defined as a constant, that is, the inertial weight of all particles (population) is the same (fixed inertial weight) throughout the iteration period. However, it is difficult to maintain the diversity of the population better by invariant inertial weight. To solve this problem, a variable inertia weight strategy is proposed, that is, the inertia weight of the population changes with the number of iterations, such as random inertia weight strategy [19], linear or non-linear inertia weight strategy [20], [21], [22], [23], [24], updating inertia weight strategy [25], [26] based on the fitness value of the optimal particle of the population at each iteration, or change inertia weight strategy of each generation by using the success rate of particle search [27]. Although these variable inertia weight strategies make the inertia weights of all particles change continuously during the iteration process, the inertia weights of all particles are the same, and the improvement of particle diversity is limited.
In order to further improve the diversity of particles in the optimization process, it is necessary to set different inertia weights for each particle in each iteration and update them continuously with the iteration process [28], that is to say, the change of particle dimension is added on the basis of variable inertia weights. According to the current research, the updating of inertia weight of particle dimension comes from the fitness value of particle itself [29], [30], [31], [32].
From the above analysis, it can be seen that the existing inertia weight improvement strategies are mainly based on single factor such as particle position (time dimension) or population optimal fitness at the last iteration, or using particle position (time axis) and particle fitness to change the inertia weight of each particle at each iteration. In fact, a particle includes m dimensions (). In previous studies, all dimensions of a particle used the same inertia weight, which led to the lack of diversity of a particle in the spatial dimension.
In addition, because there are many strategies to update inertia weight, the Ref. [33], [34] classify these strategies as: constant inertia weight, random inertia weight, time-varying inertia weight, adaptive inertia weight and so on. However, there is overlap in this classification method, for example, the random inertia weight is also time-varying inertia weight, and the adaptive inertia weight also includes time-varying inertia weight.
In this paper, on the one hand, on the basis of summarizing the current inertia weights, these inertia weight strategies are reclassified according to the changes of inertia weights in three dimensions: time (or iteration), particle and dimension (or space). On the other hand, in view of the shortcomings of the existing inertia weight improvement strategies, a multi-information fusion “triple variables with iteration” inertia weight particle swarm optimization algorithm (MFTIWPSO) is proposed, which not only uses time dimension and particle dimension, but also integrates space dimension. In order to test the optimization performance of our proposed MFTIWPSO algorithm, the benchmark function is used to test the optimization performance of MFTIWPSO, and then the algorithm is used to optimize the parameters of machine learning classifier and classify the biological datasets. The test results show that our MFTIWPSO algorithm achieves better optimization performance than other optimization algorithms.
The remainder of this paper is organized as follows. In Section 2, we briefly introduce the related research based on the improved inertial weight PSO algorithm, and propose a new classification method for the current inertial weight improvement strategy. In Section 3, we propose a multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO). In Section 4, benchmark functions and parameter settings are introduced, and in Section 5, the results are analysed. Section 6 is the comparison with high dimension benchmark function, and Section 7 is the application of the proposed PSO algorithm. Finally, in Section 8, the future work is summarized and discussed.
Section snippets
Related work
In this section, we first introduce the principle of PSO algorithm and the influence of each part of the speed update formula on the algorithm. Then, on the basis of summarizing the existing inertia weight improvement strategies, we reclassify the existing inertia weight improvement strategies according to the change of inertia weight in time (or iteration), particle and dimension (or space): the first type is “no variable with iteration” inertia weight, the second type is “single variable with
Proposed multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO)
In this section, firstly, we propose a multi-information fusion “triple variables with iteration” inertia weight method named MFTIWPSO and analyse the multi-information fusion and “triple variables with iteration” inertia weight strategy. Secondly, the influence of various factors on the improved inertia weights is analysed. Finally, the flow chart and pseudocode of the improved algorithm are given.
Benchmark functions
We use the MFTIWPSO algorithm to optimize the 22 benchmark functions in Ref. [32], and all test functions are minimum optimization. The information of each test function is listed in Table 2.
Table 2 shows the calculation formulas, dimensions, search ranges and minimum values of 22 benchmark functions. The test functions include unimodal, multimodal, rotated multimodal and composite functions. – are unimodal functions, which mainly test the optimization accuracy of the algorithm; – are
Benchmark functions test
In this section, we compare the proposed MFTIWPSO algorithm with other methods in literatures, which include: RANDPSO [19], LHNPSO [38], AIWPSO [27], DESIWPSO [28] and SAIWPSO [32]. The parameters of each algorithm are shown in Table 3.
High dimensional benchmark functions test
In order to further test the performance of the proposed MFTIWPSO algorithm, we increase the dimension of the benchmark function and other parameters are not changed. Table 8 shows the information of each function used 100 and 300 dimensions respectively.
Classification datasets
In this Section, the improved algorithm is applied to the parameter optimization of machine learning model. The datasets [34] used in this paper include Breast Cancer, Diabetes, Liver-disorders, Parkinsons, Statlog (heart) and Lung-A (lung cancer). Details of datasets are given in Table 11. Breast Cancer dataset contains 683 samples with 9 features. Diabetes dataset includes 768 samples and 8 features. Liver-disorders dataset consists of 345 samples with 6 features, and 341 samples remained
Concluding remarks
This paper summarizes the improvement strategies of inertia weight in PSO algorithm. In view of the shortcomings of the current classification methods of inertia weight, a new classification method based on the change of inertia weight in time (or iteration), particle and dimension (or space) is proposed. The improvement strategies of inertia weight are divided into “no variable with iteration”, “single variable with iteration”, “double variables with iteration”, and “triple variables with
Declaration of Competing Interest
No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.asoc.2019.105677.
Acknowledgements
This work is supported by the National Natural Science Foundation of China (61602017 and 61420106005), the National Basic Research Programme of China (2014CB744600), “Rixin Scientist” Foundation of Beijing University of Technology, China (2017-RX(1)-03), the Beijing Natural Science Foundation, China (4164080), the Beijing Outstanding Talent Training Foundation, China (2014000020124G039), the International Science and Technology Cooperation Program of China (2013DFA32180), the Special fund of
References (45)
- et al.
Grey wolf optimizer
Adv. Eng. Softw.
(2014) - et al.
On the improved performances of the particle swarm optimization algorithms with adaptive parameters, cross-over operators and root mean square (RMS) variants for computing optimal control of a class of hybrid systems
Appl. Soft Comput.
(2008) - et al.
A novel particle swarm optimization algorithm with adaptive inertia weight
Appl. Soft Comput.
(2011) - et al.
Adaptive particle swarm optimization approach for static and dynamic economic load dispatch
Energy Convers. Manag.
(2008) - et al.
A novel stability-based adaptive inertia weight for particle swarm optimization
Appl. Soft Comput.
(2016) - et al.
Low-discrepancy sequence initialized particle swarm optimization algorithm with high-order nonlinear time-varying inertia weight
Appl. Soft Comput.
(2015) - et al.
Automatic test data generation based on reduced adaptive particle swarm optimization algorithm
Neurocomputing
(2015) Seeding the initial population with feasible solutions in metaheuristic optimization of steel trusses
Eng. Optim.
(2017)Upper bound strategy for metaheuristic based design optimization of steel frames
Adv. Eng. Softw.
(2013)Enhanced hybrid metaheuristic algorithms for optimal sizing of steel truss structures with numerous discrete variables
Struct. Multidiscip. Optim.
(2017)
A grey wolf optimizer approach for optimal weight design problem of the spur gear
Eng. Optim.
Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems
Neural Comput. Appl.
Dragonfly algorithm with opposition-based learning for multilevel thresholding color image segmentation
Symmetry
Polar bear optimization algorithm: meta-heuristic with fast population movement and dynamic birth and death mechanism
Symmetry
Neutronic and thermal-hydraulic aspects of loading pattern optimization during the first cycle of VVER-1000 reactor using polar bear optimization method
Ann. Nucl. Energy
Particle swarm optimization
Forecasting financial time series volatility using particle swarm optimization trained quantile regression neural network
Appl. Soft Comput.
Analysis of effect of machining parameters during electrical discharge machining using taguchi-based multi-objective PSO
Int. J. Comput. Intell. Appl.
An interval type-2 T-S fuzzy classification system based on PSO and SVM for gender recognition
Multimedia Tools Appl.
A hybrid ACO-PSO technique for path planning
An automated hybrid approach using clustering and nature inspired optimization technique for improved tumor and tissue segmentation in magnetic resonance brain images
Appl. Soft Comput.
Improving computational efficiency of particle swarm optimization for optimal structural design
Int. J. Optim. Civil Eng.
Cited by (39)
A modified power management algorithm with energy efficiency and GHG emissions limitation for hybrid power ship system
2022, Applied EnergyCitation Excerpt :In order to provide better optimization performance, many scholars have proposed some noteworthy contributions for PSO. Li et al. [16] suggested a multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO) which increased the diversity of particles by combining the multi-dimensional information of iteration, particle and search space. Chen et al. [17] combined PSO with triangular probability density function to guarantee the inertia weight gradually decreased during the evolution process.
Trajectory control of electro-hydraulic position servo system using improved PSO-PID controller
2021, Automation in ConstructionCitation Excerpt :The basic PSO algorithm (proposed in 1995) does not have an inertia weight parameter, Shi et al. [26] presented an inertia weight into the speed update formula to balance the global and local search abilities in 1998. Li et al. [27] divided the inertia weight into four categories based on the particle and dimension. Chen et al. [28] used a triangular probability density function to built a random inertia weight.
Optimization of Fixture Number in Large Thin-Walled Parts Assembly Based on IPSO
2024, Chinese Journal of Mechanical Engineering (English Edition)Task Scheduling with Improved Particle Swarm Optimization in Cloud Data Center
2024, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)