Elsevier

Applied Soft Computing

Volume 84, November 2019, 105677
Applied Soft Computing

A multi-information fusion “triple variables with iteration” inertia weight PSO algorithm and its application

https://doi.org/10.1016/j.asoc.2019.105677Get rights and content

Highlights

  • Inertia weight has a great influence on the convergence speed and convergence precision of PSO.

  • A new classification method based on inertia weight is proposed.

  • An inertia weight strategy, which varies with time, particle and dimension, is proposed.

  • A method of multi-information fusion is used for inertia weight.

  • The proposed PSO algorithm shows good performance by simulation.

Abstract

Particle swarm optimization (PSO) has many advantages such as fewer parameters, faster convergence and easy implementation; however, it is also prone to fall into local optimum. Because inertia weight parameters can increase the diversity of particles and effectively overcome this problem, a large number of studies have done on inertia weight strategy since it was put forward, and many different improvement strategies have been put forward, but these improvement strategies still do not make full use of the multi-dimensional information of particles, resulting in limited improvement of PSO performance. In this study, on the one hand, based on the summary of current inertial weight strategies, aiming at the problems existing in the classification of inertial weight strategies, the inertial weight strategies are reclassified. According to the new classification methods, the inertial weight strategies are divided into four categories, including “no variable with iteration”, “single variable with iteration”, “double variables with iteration” and “triple variables with iteration” inertia weight. On the other hand, in view of the shortcomings of the existing inertial weight improvement strategies, this study proposes a multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO), which combines the multi-dimensional information of particle’s time (or iteration), particle and dimension (or space). In order to test the optimization performance of our proposed MFTIWPSO, the benchmark functions are used to test the optimization performance, and then the algorithm is used to optimize the parameters of machine learning classifier and classify the biological datasets. The results of two tests show that the proposed MFTIWPSO algorithm has better optimization performance than other optimization algorithms.

Introduction

With the in-depth understanding of the mechanism of biological evolution and optimization problems, many methods to solve complex optimization problems have been proposed, such as particle swarm optimization (PSO), genetic algorithm (GA), simulated annealing algorithm (SAA), ant colony optimization (ACO) and other meta-heuristic algorithms [1], [2], [3], [4], [5], [6], [7], [8], [9]. Meta-heuristic algorithm simulates the biological species and weather phenomena in nature as an optimization algorithm, which is more popular because of its fast searching speed and low computational complexity. Especially in recent years, grey wolf optimization (GWO) [4], [5], dragonfly algorithm (DA) [6], [7], polar bear optimization (PBO) [8], [9], etc., have the advantages of effective global exploration and accurate local exploitation. Compared with other optimization algorithms, PSO [10] has the advantages of simple principle, less adjustable parameters, easy implementation and better optimization performance. At present, PSO algorithm has been widely used in training of neural network, multi-objective optimization, fuzzy control system, robot path planning, image segmentation, feature selection and classifier parameter optimization [11], [12], [13], [14], [15], [16], [17].

Although particle swarm optimization (PSO) has many advantages, it also has some shortcomings, for example, it is easy to fall into local optimal solution and depends on parameters. Therefore, many scholars have improved it, including the improvement of inertia weight parameters, the improvement of the overall topology and the combination with other algorithms. Inertial weight is an important parameter in particle swarm optimization (PSO), because it can maintain the diversity of particles and effectively balance the global exploration and local exploitation of particles. Therefore, since the proposed PSO algorithm with inertial weight, many scholars have proposed many strategies to improve the inertial weight.

Inertial weight was first introduced into PSO by Shi and Eberhart in 1998 [18]. At that time, inertial weight was defined as a constant, that is, the inertial weight of all particles (population) is the same (fixed inertial weight) throughout the iteration period. However, it is difficult to maintain the diversity of the population better by invariant inertial weight. To solve this problem, a variable inertia weight strategy is proposed, that is, the inertia weight of the population changes with the number of iterations, such as random inertia weight strategy [19], linear or non-linear inertia weight strategy [20], [21], [22], [23], [24], updating inertia weight strategy [25], [26] based on the fitness value of the optimal particle of the population at each iteration, or change inertia weight strategy of each generation by using the success rate of particle search [27]. Although these variable inertia weight strategies make the inertia weights of all particles change continuously during the iteration process, the inertia weights of all particles are the same, and the improvement of particle diversity is limited.

In order to further improve the diversity of particles in the optimization process, it is necessary to set different inertia weights for each particle in each iteration and update them continuously with the iteration process [28], that is to say, the change of particle dimension is added on the basis of variable inertia weights. According to the current research, the updating of inertia weight of particle dimension comes from the fitness value of particle itself [29], [30], [31], [32].

From the above analysis, it can be seen that the existing inertia weight improvement strategies are mainly based on single factor such as particle position (time dimension) or population optimal fitness at the last iteration, or using particle position (time axis) and particle fitness to change the inertia weight of each particle at each iteration. In fact, a particle includes m dimensions (m>1). In previous studies, all dimensions of a particle used the same inertia weight, which led to the lack of diversity of a particle in the spatial dimension.

In addition, because there are many strategies to update inertia weight, the Ref. [33], [34] classify these strategies as: constant inertia weight, random inertia weight, time-varying inertia weight, adaptive inertia weight and so on. However, there is overlap in this classification method, for example, the random inertia weight is also time-varying inertia weight, and the adaptive inertia weight also includes time-varying inertia weight.

In this paper, on the one hand, on the basis of summarizing the current inertia weights, these inertia weight strategies are reclassified according to the changes of inertia weights in three dimensions: time (or iteration), particle and dimension (or space). On the other hand, in view of the shortcomings of the existing inertia weight improvement strategies, a multi-information fusion “triple variables with iteration” inertia weight particle swarm optimization algorithm (MFTIWPSO) is proposed, which not only uses time dimension and particle dimension, but also integrates space dimension. In order to test the optimization performance of our proposed MFTIWPSO algorithm, the benchmark function is used to test the optimization performance of MFTIWPSO, and then the algorithm is used to optimize the parameters of machine learning classifier and classify the biological datasets. The test results show that our MFTIWPSO algorithm achieves better optimization performance than other optimization algorithms.

The remainder of this paper is organized as follows. In Section 2, we briefly introduce the related research based on the improved inertial weight PSO algorithm, and propose a new classification method for the current inertial weight improvement strategy. In Section 3, we propose a multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO). In Section 4, benchmark functions and parameter settings are introduced, and in Section 5, the results are analysed. Section 6 is the comparison with high dimension benchmark function, and Section 7 is the application of the proposed PSO algorithm. Finally, in Section 8, the future work is summarized and discussed.

Section snippets

Related work

In this section, we first introduce the principle of PSO algorithm and the influence of each part of the speed update formula on the algorithm. Then, on the basis of summarizing the existing inertia weight improvement strategies, we reclassify the existing inertia weight improvement strategies according to the change of inertia weight in time (or iteration), particle and dimension (or space): the first type is “no variable with iteration” inertia weight, the second type is “single variable with

Proposed multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO)

In this section, firstly, we propose a multi-information fusion “triple variables with iteration” inertia weight method named MFTIWPSO and analyse the multi-information fusion and “triple variables with iteration” inertia weight strategy. Secondly, the influence of various factors on the improved inertia weights is analysed. Finally, the flow chart and pseudocode of the improved algorithm are given.

Benchmark functions

We use the MFTIWPSO algorithm to optimize the 22 benchmark functions in Ref. [32], and all test functions are minimum optimization. The information of each test function is listed in Table 2.

Table 2 shows the calculation formulas, dimensions, search ranges and minimum values of 22 benchmark functions. The test functions include unimodal, multimodal, rotated multimodal and composite functions. f1f8 are unimodal functions, which mainly test the optimization accuracy of the algorithm; f9f15 are

Benchmark functions test

In this section, we compare the proposed MFTIWPSO algorithm with other methods in literatures, which include: RANDPSO [19], LHNPSO [38], AIWPSO [27], DESIWPSO [28] and SAIWPSO [32]. The parameters of each algorithm are shown in Table 3.

High dimensional benchmark functions test

In order to further test the performance of the proposed MFTIWPSO algorithm, we increase the dimension of the benchmark function and other parameters are not changed. Table 8 shows the information of each function used 100 and 300 dimensions respectively.

Classification datasets

In this Section, the improved algorithm is applied to the parameter optimization of machine learning model. The datasets [34] used in this paper include Breast Cancer, Diabetes, Liver-disorders, Parkinsons, Statlog (heart) and Lung-A (lung cancer). Details of datasets are given in Table 11. Breast Cancer dataset contains 683 samples with 9 features. Diabetes dataset includes 768 samples and 8 features. Liver-disorders dataset consists of 345 samples with 6 features, and 341 samples remained

Concluding remarks

This paper summarizes the improvement strategies of inertia weight in PSO algorithm. In view of the shortcomings of the current classification methods of inertia weight, a new classification method based on the change of inertia weight in time (or iteration), particle and dimension (or space) is proposed. The improvement strategies of inertia weight are divided into “no variable with iteration”, “single variable with iteration”, “double variables with iteration”, and “triple variables with

Declaration of Competing Interest

No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.asoc.2019.105677.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (61602017 and 61420106005), the National Basic Research Programme of China (2014CB744600), “Rixin Scientist” Foundation of Beijing University of Technology, China (2017-RX(1)-03), the Beijing Natural Science Foundation, China (4164080), the Beijing Outstanding Talent Training Foundation, China (2014000020124G039), the International Science and Technology Cooperation Program of China (2013DFA32180), the Special fund of

References (45)

  • DörterlerM. et al.

    A grey wolf optimizer approach for optimal weight design problem of the spur gear

    Eng. Optim.

    (2019)
  • MirjaliliS.

    Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems

    Neural Comput. Appl.

    (2016)
  • BaoX. et al.

    Dragonfly algorithm with opposition-based learning for multilevel thresholding color image segmentation

    Symmetry

    (2019)
  • PołapD. et al.

    Polar bear optimization algorithm: meta-heuristic with fast population movement and dynamic birth and death mechanism

    Symmetry

    (2017)
  • NasrM.A. et al.

    Neutronic and thermal-hydraulic aspects of loading pattern optimization during the first cycle of VVER-1000 reactor using polar bear optimization method

    Ann. Nucl. Energy

    (2019)
  • KennedyJ. et al.

    Particle swarm optimization

  • PradeepkumarD. et al.

    Forecasting financial time series volatility using particle swarm optimization trained quantile regression neural network

    Appl. Soft Comput.

    (2017)
  • DeyA. et al.

    Analysis of effect of machining parameters during electrical discharge machining using taguchi-based multi-objective PSO

    Int. J. Comput. Intell. Appl.

    (2017)
  • DuY. et al.

    An interval type-2 T-S fuzzy classification system based on PSO and SVM for gender recognition

    Multimedia Tools Appl.

    (2016)
  • GigrasY. et al.

    A hybrid ACO-PSO technique for path planning

  • VishnuvarthananA. et al.

    An automated hybrid approach using clustering and nature inspired optimization technique for improved tumor and tissue segmentation in magnetic resonance brain images

    Appl. Soft Comput.

    (2017)
  • AzadS.K. et al.

    Improving computational efficiency of particle swarm optimization for optimal structural design

    Int. J. Optim. Civil Eng.

    (2013)
  • Cited by (39)

    • A modified power management algorithm with energy efficiency and GHG emissions limitation for hybrid power ship system

      2022, Applied Energy
      Citation Excerpt :

      In order to provide better optimization performance, many scholars have proposed some noteworthy contributions for PSO. Li et al. [16] suggested a multi-information fusion “triple variables with iteration” inertia weight PSO algorithm (MFTIWPSO) which increased the diversity of particles by combining the multi-dimensional information of iteration, particle and search space. Chen et al. [17] combined PSO with triangular probability density function to guarantee the inertia weight gradually decreased during the evolution process.

    • Trajectory control of electro-hydraulic position servo system using improved PSO-PID controller

      2021, Automation in Construction
      Citation Excerpt :

      The basic PSO algorithm (proposed in 1995) does not have an inertia weight parameter, Shi et al. [26] presented an inertia weight into the speed update formula to balance the global and local search abilities in 1998. Li et al. [27] divided the inertia weight into four categories based on the particle and dimension. Chen et al. [28] used a triangular probability density function to built a random inertia weight.

    • Optimization of Fixture Number in Large Thin-Walled Parts Assembly Based on IPSO

      2024, Chinese Journal of Mechanical Engineering (English Edition)
    • Task Scheduling with Improved Particle Swarm Optimization in Cloud Data Center

      2024, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    View all citing articles on Scopus
    View full text