Skip to main content
SpringerLink
Log in

Ensemble pruning of ELM via migratory binary glowworm swarm optimization and margin distance minimization

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Ensemble pruning aims at attaining an ensemble composed of less size of leaners for improving classification ability. Extreme Learning Machine (ELM) is employed as a base learner in this work, in light of its salient features, an initial pool is constructed using ELM. An ensemble composed of ELMs with better performance and diversity can make it perform the best, but the average accuracy of the whole ELMs must be decreased as the increase of diversity among them. Hence there exists a balance between the diversity and the precision of ELMs. Existing works find it via diversity measures or heuristic algorithms, which cannot find the exact tradeoff. To solve the issue, ensemble pruning of ELM via migratory binary glowworm swarm optimization and margin distance minimization (EPEMBM) is proposed utilizing the integration of the proposed migratory binary glowworm swarm optimization (MBGSO) and margin distance minimization (MDM). First, the created ELMs in a pool can be pre-pruned by MDM, and it can markedly downsize the ELMs in the pool, and significantly alleviates its computation overhead. Second, the retaining ELMs are further pruned utilizing MBGSO, and the final ensemble is attained with a high efficiency. Experimental results on 21 UCI classification tasks indicate that EPEMBM outperforms techniques, and that its effectiveness and efficiency. It is a very useful tool for solving the selection problem of ELMs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Silva FAOD, Zhao L (2016) A network of neural oscillators for fractal pattern recognition. Neural Process Lett 44(1):149–159

    Google Scholar 

  2. Tripathi BK (2017) On the complex domain deep machine learning for face recognition. Appl Intell 47(3):1–15

    MathSciNet  Google Scholar 

  3. Aburomman AA, Reaz MBI (2017) A survey of intrusion detection systems based on ensemble and hybrid classifiers. Comput Secur 65:135–152

    Google Scholar 

  4. Kavitha B, Karthikeyan S, Maybell PS (2012) An ensemble design of intrusion detection system for handling uncertainty using Neutrosophic Logic Classifier. Knowl Based Syst 28:88–96

    Google Scholar 

  5. Galar M, Fernandez A, Barrenechea E et al (2016) Ordering-based pruning for improving the performance of ensembles of classifiers in the framework of imbalanced datasets. Inf Sci 354:178–196

    Google Scholar 

  6. Zhou H, Zhao X, Wang X (2014) An effective ensemble pruning algorithm based on frequent patterns. Knowl Based Syst 56:79–85

    Google Scholar 

  7. Ding S, Chen Z, Zhao S et al (2018) Pruning the ensemble of ANN based on decision tree induction. Neural Process Lett 48(1):53–70

    Google Scholar 

  8. Xia X, Lin T, Chen Z (2018) Maximum relevancy maximum complementary based ordered aggregation for ensemble pruning. Appl Intell 48(9):2568–2579

    Google Scholar 

  9. Martínez-Muñoz G, Hernández-Lobato D, Suárez A (2009) An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans Pattern Anal Mach Intell 31(2):245–259

    Google Scholar 

  10. Dai Q, Zhang T, Liu N (2015) A new reverse reduce-error ensemble pruning algorithm. Appl Soft Comput 28:237–249

    Google Scholar 

  11. Wang G, Shi N, Shu Y et al (2016) Embedded manifold-Based kernel fisher discriminant analysis for face recognition. Neural Process Lett 43(1):1–16

    Google Scholar 

  12. Bashbaghi S, Granger E, Sabourin R et al (2017) Dynamic ensembles of exemplar-SVMs for still-to-video face recognition. Pattern Recognit 69:61–81

    Google Scholar 

  13. Zhao X, Deng N, Jing L et al (2017) Application of image recognition in civil aviation security based on tensor learning. J Intell Fuzzy Syst 33(4):2145–2157

    Google Scholar 

  14. Mori S (2017) Deep architecture neural network-based real-time image processing for image-guided radiotherapy. Phys Med 40:79–87

    Google Scholar 

  15. Antipov G, Baccouche M, Berrani SA et al (2017) Effective training of convolutional neural networks for face-based gender and age prediction. Pattern Recognit 72:15–26

    Google Scholar 

  16. Li K, Xing JL, Hu WM et al (2017) D2C: deep cumulatively and comparatively learning for human age estimation. Pattern Recognit 66:95–105

    Google Scholar 

  17. Je HM, Kim D, Bang SY (2003) Human face detection in digital video using SVM ensemble. Neural Process Lett 17(3):239–252

    Google Scholar 

  18. Termenon M, Graña M (2012) A two stage sequential ensemble applied to the classification of Alzheimer’s disease based on MRI features. Neural Process Lett 35(1):1–12

    Google Scholar 

  19. Kavitha B, Karthikeyan S, Maybell PS (2012) An ensemble design of intrusion detection system for handling uncertainty using neutrosophic logic classifier. Knowl Based Syst 28:88–96

    Google Scholar 

  20. Ykhlef H, Bouchaffra D (2017) An efficient ensemble pruning approach based on simple coalitional games. Inf Fusion 34:28–42

    Google Scholar 

  21. Bi Y (2012) The impact of diversity on the accuracy of evidential classifier ensembles. Int J Approx Reason 53(4):584–607

    MathSciNet  Google Scholar 

  22. Yang C, Yin XC, Hao HW et al (2014) Classifier ensemble with diversity: effectiveness analysis and ensemble optimization. Acta Autom Sin 40(4):660–674

    MathSciNet  Google Scholar 

  23. Li N, Yu Y, Zhou ZH (2012) Diversity regularized ensemble pruning. In: Machine learning and knowledge discovery in databases, 2012, pp 330–345

  24. Tang EK, Suganthan PN, Yao X (2006) An analysis of diversity measures. Mach Learn 65:247–271

    Google Scholar 

  25. Dai Q, Ye R, Liu Z (2017) Considering diversity and accuracy simultaneously for ensemble pruning. Appl Soft Comput 58:75–91

    Google Scholar 

  26. Partalas I, Tsoumakas G, Vlahavas I (2009) Pruning an ensemble of classifiers via reinforcement learning. Neurocomputing 72:1900–1909

    Google Scholar 

  27. Martínez-Muñoz G, Suárez A (2004) Aggregation ordering in bagging. In: Proceedings of the IASTED international conference on artificial intelligence and applications, 2004, pp 258–263

  28. Krishnanand KN, Ghose D (2009) Glowworm swarm optimisation: a new method for optimising multi-modal functions. Int J Comput Intell Stud 1(1):93–119

    Google Scholar 

  29. Krishnanand KN, Ghose D (2009) Glowworm swarm optimization for simultaneous capture of multiple local optima of multimodal functions. Swarm Intell 3(2):87–124

    Google Scholar 

  30. Krishnanand KN, Ghose D (2006) Glowworm swarm based optimization algorithm for multimodal functions with collective robotics applications. Multiagent Grid Syst 2(3):209–222

    MATH  Google Scholar 

  31. Li MW, Wang X, Gong Y (2014) Binary glowworm swarm optimization for unit commitment. J Mod Power Syst Clean Energy 2(4):357–365

    Google Scholar 

  32. Zhou YQ, Huang ZX, Liu HX (2012) Discrete glowworm swarm optimization algorithm for TSP problem. Acta Electron Sin 40(6):1164–1170

    Google Scholar 

  33. Martínez-Muñoz G, Suárez A (2006) Pruning in ordered bagging ensembles. In: Proceedings of the twenty-third international conference on machine learning, 2006, pp 609–616

  34. Margineantu DD, Dietterich TG (1997) Pruning adaptive boosting. In: Proceedings of the fourteenth international conference on machine learning, vol 97, 1997, pp 211–218

  35. Lu Z, Wu X, Zhu X et al (2010) Ensemble pruning via individual contribution ordering. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 871–880

  36. Guo L, Boukir S (2013) Margin-based ordered aggregation for ensemble pruning. Pattern Recognit Lett 34:603–609

    Google Scholar 

  37. Guo H, Liu H, Li R et al (2018) Margin and diversity based ordering ensemble pruning. Neurocomputing 275:237–246

    Google Scholar 

  38. Zhou ZH, Wu J, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1):239–263

    MathSciNet  MATH  Google Scholar 

  39. Ni ZW, Zhang C, Ni LP (2016) Haze forecast method of selective ensemble based on glowworm swarm optimization algorithm. Pattern Recognit Artif Intell 29(2):143–153

    MathSciNet  Google Scholar 

  40. Rokach L (2009) Collective-agreement-based pruning of ensembles. Comput Stat Data Anal 53(4):1015–1026

    MathSciNet  MATH  Google Scholar 

  41. De Oliveira JV, Alexandre S, De Castro LN (2017) Particle Swarm Clustering in clustering ensembles: Exploiting pruning and alignment free consensus. Appl Soft Comput 55:141–153

  42. Bai L, Liang J, Cao F (2020) A multiple k-means clustering ensemble algorithm to find nonlinearly separable clusters. Inf Fusion 61:36–47

  43. Akbari E , Dahlan H, Ibrahim R (2015) Hierarchical cluster ensemble selection. Eng Appl Artif Intell 39:146–156

    Google Scholar 

  44. Giacinto G, Roli F, Fumera G (2000) Design of effective multiple classifier systems by clustering of classifiers. In: International conference on pattern recognition, 2000, pp 160–163

  45. Lu HJ, An CL, Zheng EH, Lu Y (2014) Dissimilarity based ensemble of extreme learning machine for gene expression data classification. Neurocomputing 128:22–30

    Google Scholar 

  46. Cavalcanti GDC, Oliveira LS, Moura TJM et al (2016) Combining diversity measures for ensemble pruning. Pattern Recognit Lett 74:38–45

    Google Scholar 

  47. Ding SF, An YX, Zhang XK et al (2017) Wavelet twin support vector machines based on glowworm swarm optimization. Neurocomputing 225:157–163

    Google Scholar 

  48. Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36(3):285–305

    Google Scholar 

  49. Breiman L (1996) Bagging predictors. Mach Learn 24:123–140

    MATH  Google Scholar 

  50. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489–501

    Google Scholar 

  51. Zhang Y, Wu J, Cai Z et al (2016) Memetic extreme learning machine. Pattern Recognit 58:135–148

    Google Scholar 

  52. Cai Y, Liu X, Zhang Y et al (2018) Hierarchical ensemble of extreme learning machine. Pattern Recognit Lett 116:101–106

    Google Scholar 

  53. Azad MAK, Rocha AMAC, Fernandes EMGP (2014) Improved binary artificial fish swarm algorithm for the 0–1 multidimensional knapsack problems. Swarm Evol Comput 14:66–75

    Google Scholar 

  54. Singhal PK, Naresh R, Sharma V (2015) Binary fish swarm algorithm for profit-based unit commitment problem in competitive electricity market with ramp rate constraints. IET Gener Transm Distrib 9(13):1697–1707

    Google Scholar 

  55. RezaeeJordehi A (2019) Binary particle swarm optimisation with quadratic transfer function: a new binary optimisation algorithm for optimal scheduling of appliances in smart homes. Appl Soft Comput 78:465–480

    Google Scholar 

  56. Islam MJ, Li X, Mei Y (2017) A time-varying transfer function for balancing the exploration and exploitation ability of a binary PSO. Appl Soft Comput 59:185–196

    Google Scholar 

  57. Patle BK, Parhi DRK, Jagadeesh A et al (2018) Matrix-binary codes based genetic algorithm for path planning of mobile robot. Comput Electr Eng 67:708–728

    Google Scholar 

  58. Das AK, Sengupta S, Bhattacharyya S (2018) A group incremental feature selection for classification using rough set theory based genetic algorithm. Appl Soft Comput 65:400–411

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Anhui Provincial Natural Science Foundation under Grant No. 1908085QG298, the National Nature Science Foundation of China under Grant No. 91546108, the Fundamental Research Funds for the Central Universities Nos. JZ2019HGTA0053, JZ2019HGBZ0128, the Anhui Provincial Science and Technology Major Projects No. 201903a05020020, and the Open Research Fund Program of Key Laboratory of Process Optimization and Intelligent Decision-making (Hefei University of Technology), Ministry of Education. In addition, we thank Pingfan Xia for the help in some experiments.

Author information

Authors and Affiliations

  1. School of Management, Hefei University of Technology, Hefei, 230009, China

    Xuhui Zhu, Zhiwei Ni, Liping Ni, Feifei Jin & Zhangjun Wu

  2. Key Laboratory of Process Optimization and Intelligent Decision-Making, Ministry of Education, Hefei, 230009, China

    Xuhui Zhu, Zhiwei Ni, Liping Ni, Feifei Jin & Zhangjun Wu

  3. Business School, Huzhou University, Huzhou, 313000, China

    Meiying Cheng

Authors
  1. Xuhui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiwei Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liping Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feifei Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meiying Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhangjun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiwei Ni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Ni, Z., Ni, L. et al. Ensemble pruning of ELM via migratory binary glowworm swarm optimization and margin distance minimization. Neural Process Lett 52, 2043–2067 (2020). https://doi.org/10.1007/s11063-020-10336-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10336-2

Keywords

Search

Navigation