Efficient multi-population outpost fruit fly-driven optimizers: Framework and advances in support vector machines

https://doi.org/10.1016/j.eswa.2019.112999Get rights and content

Highlights

  • Outpost mechanism and multi-swarm mechanism are introduced into FOA.

  • The enhanced FOA is used for constructing an evolutionary support vector machine.

  • Classic and CEC benchmark problems are used to verify the proposed method.

  • The proposed method is also applied for predicting the financial stress problems.

Abstract

The original fruit fly algorithm (FOA) in simple structure is easy to understand, but it has a slow convergence rate and tends to be trapped in the local optimal solutions. In order to improve the convergence rate and efficacy of FOA, two new mechanisms are integrated with the exploratory and exploitative strategies of the original FOA: the outpost mechanism and the multi-population mechanism. The outpost mechanism consists of two parts: greedy selection and Gaussian mutation, which is mainly used to improve the convergence rate of the algorithm. The multi-swarm mechanism divides the population of agents into several sub-swarms and selects several individuals from sub-swarm with a random probability. Then, the selected individuals are remapped into the feature space to expand the exploratory capabilities. To illustrate the performance of the proposed method, a comprehensive set of benchmark functions, including the unimodal, multimodal, and composition functions were chosen for testing tasks. Also, the proposed MOFOA is compared against the state-of-the-art improved FOA algorithms and other well-known swarm-based methods. The experimental results have shown that MOFOA can outperform all the competitors involved in this study in terms of convergence speed and solution quality in a significant manner. Furthermore, MOFOA is also employed to optimize two critical parameters of the support vector machine (SVM) for classification tasks. The results demonstrate that the proposed MOFOA can also achieve a better performance than other swarm-based methods in dealing with the optimization of the SVM in dealing with several financial datasets.

Introduction

Traditional gradient-based methods often have great difficulties and limitations in dealing with global optimization problems with high nonlinearity and multiple local optimums (Zhang, Wang, Zhou & Ma, 2019a). In recent decades, a large number of meta-heuristic optimization algorithms with various characteristics have been proposed (Chen, Xu, Wang & Zhao, 2019; Deng et al., 2017a; Deng et al., 2017b; Deng, Xu & Zhao, 2019; Luo et al., 2018; Shen et al., 2016; Yu, Zhao, Wang, Chen & Li, 2019; Zhang et al., 2018a; Zhang, Hu, Qu & Maybank, 2010; Zhang et al., 2019a; Zhao et al., 2019). Metaheuristics are widely used to deal with these global optimization problems because of their simplicity, high efficiency, and low computational cost. Among the more classic algorithms are: firefly algorithm (FA) (Yang, 2009), moth-flame optimization algorithm (MFO) (Mirjalili, 2015), particle swarm optimization (PSO) (Kennedy & Eberhart, 1995), different evolution (DE) (Storn & Price, 1997), bat algorithm (BA) (Yang 2010a, Yang, 2010b), sine cosine algorithm (SCA) (Mirjalili, 2016b), multi-verse optimization algorithm (MVO) (Mirjalili, Mirjalili & Hatamlou, 2016), dragonfly algorithm (DA) (Mirjalili, 2016a), whale optimization algorithm (WOA) (Mirjalili & Lewis, 2016), grey wolf optimization (GWO) (Mirjalili, Mirjalili & Lewis, 2014), Harris hawks optimizer (HHO) (Heidari et al., 2019), salp swarm algorithm (SSA) (Mirjalili et al., 2017). Fruit fly optimization algorithm (FOA) (Pan, 2012) is a recent swarm intelligence algorithm based on the simulation of fruit fly using visual and olfactory foraging, which was proposed by Pan in 2012. However, as no free lunch (NFL) theorem states, no optimizer can be treated as a universally best winner for all kind of problems (Wolpert & Macready, 1997). Hence, there are a wide variety of evolutionary and swarm-based optimization algorithms in the literature. The fact is that a good optimizer may even obtain exact optimum results for a problem. However, it may also get converged to local optima or wrong solutions in dealing with another class of problems. For instance, an evolutionary and swarm-based optimizer may show an excellent performance for continuous function optimization problems, but provide low-quality solutions for a binary feature selection scenario. Performance of swarm-based optimizers may not always be stable, and they may fail in balancing between its central stochastic exploratory and exploitative tendencies on some occasions. This fact does not mean that we need always to develop a new optimizer from scratch to reach to more high-quality solutions. The results of NFL theorem encourage us to modify further the operations of an existing optimizer for particular classes of problems to alleviate its core shortcomings and reach a more stable variant. Hence, even with the existence of alternative meta-heuristics, we modify the well-established FOA to reach more stable performance and a more efficient variant. To provide a more stable optimizer and classifier with higher accuracy rates for data classification and prediction in the financial field, this paper fuses the support vector machines (SVM) and boosted FOA with the outpost and the multi-population mechanisms to propose a hybrid efficient model called MOFOA-SVM with substantial advantages in dealing with the classification of financial datasets.

As shown in Table 1, there are fewer control parameters of FOA compared with other biomimetic optimization algorithms (Deng et al., 2019; Heidari et al., 2019; Luo et al., 2019; Xu et al., 2019a, Xu et al. 2019b). It is uncomplicated and straightforward to implement; therefore, it has been applied to many fields of optimization, such as feature selection(Appavu & Nancy, 2018), artificial neural network optimization (Liang, Niu & Hong, 2019), structural optimization (Ke, Zhang, Li & Du, 2016), optimization of operating costs (Xiao & Chen, 2016), etc. However, as FOA emphasizes the search for local optimal values, the performance cannot be fully explored with a certain probability to be trapped in local optimum, especially when the dimension of the problems is high. Therefore, many researchers have tried to improve this algorithm in recent years. Du, Ke, Liao & Shen (2018) used stepwise linear decrement and Logistic chaotic mapping to deal with the non-binding optimization problem of the benchmark function and applied it to the constrained structural engineering design optimization problem. Pan, Sang, Duan & Gao (2014) used a new control parameter to adjust the search radius of its population dynamically and then used the algorithm to solve the continuity function optimization problem.

Shan, Cao & Dong (2013) proposed an improved FOA based on linear generation mechanism of candidate solution (LGMS). By dynamically adjusting the search step size with the change of iterations, the efficiency and reliability of the FOA were improved a lot. Ye, Lou & Sun (2017) proposed to initialize the position of the fruit fly population by chaotic particles and replaced the distance expression of the fruit flies looking for food sources. The proposed mutation strategy used two different generation mechanisms in the olfactory phase so that the algorithm can search for the local optimal solution in the entire solution space. Wang & Liu (2015) used the odor concentration variable τ2 to indicate changes in all individual fitness values and odor concentration values in the current environment to determine whether they have fallen into a locally optimal solution. If trapped, adaptive mutation FOA (AM-FOA) selected an appropriate number of fruit flies to mutate with a certain probability. Si et al. (2016) proposed to set the odor concentration determination value to a positive or negative value with a random probability and used two different position update methods before or after the iteration.

Li, Geng, Han & Zheng (2016) proposed an improved FOA algorithm, CAEFOA. The CAEFOA defined an adaptive adjustment parameter and used the adaptive flight guidance rule to make the population search for a better solution near the current optimal solution space. At the same time, the population traverses the feasible region through the global chaotic perturbation mechanism. Yuan, Dai, Zhao and He (2014) proposed a multi-group optimization mechanism MFOA, in which multiple sub-groups search in the solution space to find the global optimal solution. At the same time, the step size of the subgroup decreases with the number of iterations. Zhang, Lu, Jia & Li (2018b) proposed a θ-MAFOA which combined phase angle coding with mutation adaptation mechanism to improve the convergence speed and the exploration ability of the algorithm. Kanarachos, Dizqah, Chrysakis & Fitzpatrick (2018) improved the fruit fly by adding two conditions. When the first condition did not find the optimal solution, it entered the decision delay phase and then chose to enter the reset phase, the casting phase, or the visual feature detection phase. Zheng & Wang (2016) proposed an adaptive two-stage fruit fly optimization algorithm, TAFOA. In the first phase, a high-quality initial solution is generated by the heuristic search. In the second phase, the initial solution is used as the initial cluster center for continuous evolution, and the knowledge orientation of specific problems is used as a search method to evolve.

Wang, Shi & Liu (2015) introduced swarm collaboration in the algorithm so that some fruit flies continued to search for things randomly after the population found food. Then the parameter ω was introduced to make the population mutate the random disturbance, which improved the ability of the algorithm to jump out of the local optimal solution. Niu, Zhong, Liang, Luo & Qian (2015) used the expression of the odor concentration determination value to distribute the Distij in solution space in a standard way and exerted the differential evolution operation instead of the random search. Han, Liu, Wang & Wang (2018) used two mechanisms in the proposed CEFOA: the trend search strategy was used to evaluate the quality of the solution; the co-evolution mechanism determined whether the population was near the local optimal solution through the self-judgment process. If the probability of premature convergence exceeded the threshold, the flight was escaped by this mechanism to improve the global search ability and the diversity of the search direction of the algorithm. The IAFOA proposed by Wu, Liu, Tian, Zhang & Xiao (2018) contained four additional mechanisms: (1) adaptive selection mechanism for search direction (2) adaptive adjustment mechanism for iterative step size (3) adaptive cross-variation mechanism (4) multi-subgroup mechanism. The multi-scale cooperative mutation mechanism in MSFOA proposed by Zhang, Cui, Wu, Pan & He (2016) used Gaussian mutations with different proportional differences to escape local optimum. Hu, Wen, Zeng & Huang (2017) changed the fixed step size to the step size that decreased with the number of iterations to improve the accuracy of FOA. Darvish & Ebrahimzadeh (2018) introduced the alternative search engine to make the search engine related to the maximum number of iterations and introduced a new average engine to improve the efficiency and stability of the algorithm in high-dimensional problems. The similarities and features of the improved FOA were sorted out in Table 2.

Although many researchers proposed several improved versions of FOA, the algorithm and its enhanced versions are still prone to some significant performance problems. They may trap into local optima and face the stagnation shortcoming, quickly. Given the above problems, this paper proposed a multi-population optimization FOA (MOFOA) with the outpost mechanism. Two mechanisms, including the outpost and the multi-population mechanisms, were embedded in the conventional FOA. The Outpost mechanism significantly improved the convergence efficiency of the algorithm through the greedy selection and Gaussian variation. Multi-population mechanism divided the population into multiple subpopulations, each of them independently searches within the feature space. Some individuals in each sub-group have a specific chance to be redistributed into the solution space and contribute to enhancing the diversity of the population. This strategy can avoid the population from falling into the local optimal and improve the convergence accuracy of the algorithm. In order to verify the performance of MOFOA, the Wilcoxon signed-rank test method and the Freidman test were used for comparing the algorithm with other competitors including modern, improved variants, and advanced algorithms based on 23 well-regarded functions and problems from IEEE CEC2014 test suite. The results show that the performance of MOFOA has been improved compared to other competitors.

This paper validates and combines the advanced MOFOA with SVM paradigm to introduce a new classification model named MOFOA-SVM and uses it to tackle the prediction tasks in the financial field, such as bankruptcy prediction of enterprises or banks. The parameters that determine the classification performance of the SVM model are the disciplinary factor C and the kernel width γ, which can be well optimized by MOFOA. In order to reflect the classification performance of MOFOA-SVM, this paper compared MOFOA-SVM with four other classification models (FOA-SVM, WOA-SVM, GWO-SVM, and SCA-SVM) in terms of four performance metrics including Accuracy (ACC), Specificity, Sensitivity, Matthews Correlation Coefficients (MCC) and also conducted 10-fold cross-validation analysis (Salzberg, 1997). The experimental results demonstrated that the classification prediction performance of the MOFOA-SVM model has distinct advantages compared to other classifiers.

The parts of this research are organized as follows: Section 2 introduced the specific background of the fruit fly optimization algorithm. Section 3 explained the two new mechanisms embedded in FOA in detail. In Section 4, a series of comparison experiments were carried out on the MOFOA. Section 6 summarized the work of the whole chapter and pointed out the directions of future work.

Section snippets

Fruit fly optimization algorithm

Many real-life problems can be realized and solved by using proper mathematical modeling (Gao, Guirao, Abdel-Aty & Xi, 2019;; Gao et al., 2018a; Gao et al., 2018b; Gao, Wu, Siddiqui & Baig, 2018; Wei, Darko & Hosam, 2018). Stochastic nature-inspired methods try to simulate phenomena in nature to develop an optimization model that can solve a problem, iteratively (Aljarah, Mafarja, Heidari, Faris & Mirjalili, 2019; Mafarja et al., 2018; Xu et al., 2019a). Pan first proposed FOA in 2012 (Pan, 2012

Proposed MOFOA

The original FOA has the advantages of simple structure, few control parameters, and easy understanding. However, the convergence speed and accuracy of the classical FOA on multi-mode functions and complex functions are still not ideal. Given the above shortcomings, this paper proposes two improved mechanisms to improve the convergence speed and solution quality of the classical FOA.

Experiments and results

In this section, we quantitatively analyzed the multi-population mechanism and then compared the proposed MOFOA with traditional metaheuristic algorithms, improved FOA algorithms, and modified metaheuristic algorithms in a comprehensive set of 20 benchmark functions. All experiments were conducted on Windows Server 2008 R2 operating system with Intel (R) Xeon (R) CPU E5-2660 v3 (2.60 GHz) and 16 GB of RAM. All algorithms mentioned above were coded on the MATLAB R2014b.

In order to compare

Discussions

From the previous comprehensive experimental results, MOFOA showed better convergence performance in unimodal functions F1-F7, which was due to the improvement of convergence speed by outpost mechanisms. Outpost mechanism enables fruit flies to appear at higher odor concentration with a certain probability through position variation based on the normal distribution and then updates the current optimal location by greedy selection to achieve better fitness with relative fewer evaluation times.

Conclusions and future directions

In this study, an enhanced MOFOA algorithm with the outpost mechanism and the multi-population mechanism was proposed. In MOFOA, the outpost mechanism significantly improves the convergence rate of the algorithm in the early stage. Also, the multi-population mechanism improves global searching capability. Hence, it assists the algorithm in avoiding falling into local optimum to some extent. The proposed MOFOA was compared with a comprehensive set of metaheuristic algorithms, including the base

CRediT authorship contribution statement

Huiling Chen: Conceptualization, Methodology, Software, Writing - original draft, Visualization, Writing - review & editing. Shimin Li: Writing - original draft, Writing - review & editing, Software, Visualization, Investigation. Ali Asghar Heidari: Writing - original draft, Writing - review & editing, Software, Visualization, Investigation. Pengjun Wang: Writing - review & editing, Software, Visualization. Jiawei Li: Writing - review & editing, Software, Visualization. Yutao Yang:

Declaration of Competing Interest

The authors declare that there is no conflict of interests regarding the publication of article.

Acknowledgment

This research is supported by Zhejiang Provincial Natural Science Foundation of China (LJ19F020001), Key R&D Program Projects in Zhejiang Province (2019C01041), Science and Technology Plan Project of Wenzhou, China (2018ZG012, ZG2017019), and National Natural Science Foundation of China (U1809209).

References (76)

  • R. Hu et al.

    A short-term power load forecasting model based on the generalized regression neural network with decreasing step fruit fly optimization algorithm

    Neurocomputing

    (2017)
  • D. Jia et al.

    A hybrid particle swarm optimization algorithm for high-dimensional problems

    Computers and Industrial Engineering

    (2011)
  • S. Kanarachos et al.

    Optimal design of a quadratic parameter varying vehicle suspension system using contrast-based fruit fly optimisation

    Applied Soft Computing Journal

    (2018)
  • M.W. Li et al.

    Ship motion prediction using dynamic seasonal RvSVR with phase space reconstruction and the chaos adaptive efficient FOA

    Neurocomputing

    (2016)
  • Y. Liang et al.

    Short term load forecasting based on feature extraction and improved general regression neural network model

    Energy

    (2019)
  • J. Luo et al.

    Multi-strategy boosted mutative whale-inspired optimization approaches

    Applied Mathematical Modelling

    (2019)
  • J. Luo et al.

    An improved grasshopper optimization algorithm with application to financial stress prediction

    Applied Mathematical Modelling

    (2018)
  • M. Mafarja et al.

    Binary dragonfly optimization for feature selection using time-varying transfer functions

    Knowledge-Based Systems

    (2018)
  • S. Mirjalili

    Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm

    Knowledge-Based Systems

    (2015)
  • S. Mirjalili

    SCA: A sine cosine algorithm for solving optimization problems

    Knowledge-Based Systems

    (2016)
  • S. Mirjalili et al.

    Salp swarm algorithm: A bio-inspired optimizer for engineering design problems

    Advances in Engineering Software

    (2017)
  • S. Mirjalili et al.

    The whale optimization algorithm

    Advances in Engineering Software

    (2016)
  • S. Mirjalili et al.

    Grey wolf optimizer

    Advances in Engineering Software

    (2014)
  • J. Niu et al.

    Fruit fly optimization algorithm based on differential evolution and its application on gasification process operation optimization

    Knowledge-Based Systems

    (2015)
  • Q.K. Pan et al.

    An improved fruit fly optimization algorithm for continuous function optimization problems

    Knowledge-Based Systems

    (2014)
  • W.T. Pan

    A new fruit fly optimization algorithm: Taking the financial distress model as an example

    Knowledge-Based Systems

    (2012)
  • P. Ravi Kumar et al.

    Bankruptcy prediction in banks and firms via statistical and intelligent techniques - A review

    European Journal of Operational Research

    (2007)
  • L. Shen et al.

    Evolving support vector machines using fruit fly optimization for medical data classification

    Knowledge-Based Systems

    (2016)
  • L. Wang et al.

    An improved fruit fly optimization algorithm and its application to joint replenishment problems

    Expert Systems with Applications

    (2015)
  • W. Wang et al.

    Melt index prediction by least squares support vector machines with an adaptive mutation fruit fly optimization algorithm

    Chemometrics and Intelligent Laboratory Systems

    (2015)
  • L. Wu et al.

    A new improved fruit fly optimization algorithm IAFOA and its application to solve engineering optimization problems

    Knowledge-Based Systems

    (2018)
  • Y. Xu et al.

    An efficient chaotic mutative moth-flame-inspired optimizer for global optimization tasks

    Expert Systems with Applications

    (2019)
  • Y. Xu et al.

    Enhanced moth-flame optimizer with mutation strategy for global optimization

    Information Sciences

    (2019)
  • X. Yuan et al.

    On a novel multi-swarm fruit fly optimization algorithm and its application

    Applied Mathematics and Computation

    (2014)
  • X Yuan et al.

    On a novel multi-swarm fruit fly optimization algorithm and its application

    Applied Mathematics and Computation

    (2014)
  • X. Zhang et al.

    A novel phase angle-encoded fruit fly optimization algorithm with mutation adaptation mechanism applied to UAV path planning

    Applied Soft Computing Journal

    (2018)
  • Y. Zhang et al.

    A novel multi-scale cooperative mutation fruit fly optimization algorithm

    Knowledge-Based Systems

    (2016)
  • X. Zhao et al.

    Chaos enhanced grey wolf optimization wrapped ELM for diagnosis of paraquat-poisoned patients

    Computational Biology and Chemistry

    (2019)
  • Cited by (93)

    View all citing articles on Scopus
    View full text