Skip to main content
SpringerLink
Log in

Ensemble Learning Based on Multimodal Multiobjective Optimization

  • Conference paper
  • First Online:
Bio-inspired Computing: Theories and Applications (BIC-TA 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1159))

Abstract

In ensemble learning, the accuracy and diversity are two conflicting objectives. As the number of base learners increases, the prediction speed of ensemble learning machines drops significantly and the required storage space also increases rapidly. How to balance these two goals for selective ensemble learning is an extremely essential problem. In this paper, ensemble learning based on multimodal multiobjective optimization is studied in detail. The great significance and importance of multimodal multiobjective optimization algorithm is to find these different classifiers ensemble by considering the balance between accuracy and diversity, and different classifiers ensemble correspond to the same accuracy and diversity. Experimental results show that multimodal multiobjective optimization algorithm can find more ensemble combinations than unimodal optimization algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Li, W., Ding, S., Wang, H., Chen, Y., Yang, S.: Heterogeneous ensemble learning with feature engineering for default prediction in peer-to-peer lending in China. World Wide Web 23, 23–45 (2020). https://doi.org/10.1007/s11280-019-00676-y

    Article  Google Scholar 

  2. Barushka, A., Hajek, P.: Spam filtering in social networks using regularized deep neural networks with ensemble learning. In: Iliadis, L., Maglogiannis, I., Plagianakos, V. (eds.) AIAI 2018. IAICT, vol. 519, pp. 38–49. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92007-8_4

    Chapter  Google Scholar 

  3. Bekiroglu, K., Duru, O., Gulay, E., Su, R., Lagoa, C.: Predictive analytics of crude oil prices by utilizing the intelligent model search engine. Appl. Energy 228, 2387–2397 (2018)

    Article  Google Scholar 

  4. Lemaître, G., Nogueira, F., Aridas, C.K.: Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J. Mach. Learn. Res. 18(1), 559–563 (2017)

    MATH  Google Scholar 

  5. Oh, S., Lee, M.S., Zhang, B.T.: Ensemble learning with active example selection for imbalanced biomedical data classification. IEEE/ACM Trans. Comput. Biol. Bioinf. 8(2), 316–325 (2010)

    Google Scholar 

  6. Zhang, C., Ma, Y.: Ensemble Machine Learning: Methods and Applications. Springer, Boston (2012). https://doi.org/10.1007/978-1-4419-9326-7

    Book  MATH  Google Scholar 

  7. Zhou, Z.H., Wu, J., Wei, T.: Ensembling neural networks: many could be better than all. Artif. Intell. 137(1), 239–263 (2002)

    Article  MathSciNet  Google Scholar 

  8. Islam, M.M., Xin, Y.: Evolving artificial neural network ensembles. IEEE Comput. Intell. Mag. 3(1), 31–42 (2008)

    Article  Google Scholar 

  9. Pan, L., He, C., Tian, Y., Wang, H., Zhang, X., Jin, Y.: A classification-based surrogate-assisted evolutionary algorithm for expensive many-objective optimization. IEEE Trans. Evol. Comput. 23(1), 74–88 (2018)

    Article  Google Scholar 

  10. Pan, L., He, C., Tian, Y., Su, Y., Zhang, X.: A region division based diversity maintaining approach for many-objective optimization. Integr. Comput. Aided Eng. 24(3), 279–296 (2017)

    Article  Google Scholar 

  11. He, C., Tian, Y., Jin, Y., Zhang, X., Pan, L.: A radial space division based evolutionary algorithm for many-objective optimization. Appl. Soft Comput. 61, 603–621 (2017)

    Article  Google Scholar 

  12. Pan, L., Li, L., He, C., Tan, K.C.: A subregion division-based evolutionary algorithm with effective mating selection for many-objective optimization. IEEE Trans. 2019, 1–14 (2019)

    Google Scholar 

  13. Bui, L.T., Dinh, T.T.H., et al.: A novel evolutionary multi-objective ensemble learning approach for forecasting currency exchange rates. Data Knowl. Eng. 114, 40–66 (2018)

    Article  Google Scholar 

  14. Ojha, V.K., Abraham, A., Snášel, V.: Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming. Appl. Soft Comput. 52, 909–924 (2017)

    Article  Google Scholar 

  15. Mocanu, D.C., Mocanu, E., Stone, P., Nguyen, P.H., Gibescu, M., Liotta, A.: Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat. Commun. 9(1), 2383 (2018). https://doi.org/10.1038/s41467-018-04316-3

    Article  Google Scholar 

  16. Hu, J., Li, T., Luo, C., Fujita, H., Yang, Y.: Incremental fuzzy cluster ensemble learning based on rough set theory. Knowl.-Based Syst. 132, 144–155 (2017)

    Article  Google Scholar 

  17. Yang, D., Liu, Y., Li, S., Li, X., Ma, L.: Gear fault diagnosis based on support vector machine optimized by artificial bee colony algorithm. Mech. Mach. Theory 90, 219–229 (2015)

    Article  Google Scholar 

  18. Ni, Z., Zhang, C., Ni, L.: Haze forecast method of selective ensemble based on glowworm swarm optimization algorithm. Int. J. Pattern Recognit. Artif. Intell. 29(2), 143–153 (2016)

    MathSciNet  Google Scholar 

  19. Yong, Z., Bo, L., Fan, Y.: Differential evolution based selective ensemble of extreme learning machine. In: Trustcom/BigDataSE/ISPA, pp.1327–1333 (2017)

    Google Scholar 

  20. Liu, Y., Yao, X., Higuchi, T.: Evolutionary ensembles with negative correlation learning. IEEE Trans. Evol. Comput. 4(4), 380–387 (2000)

    Article  Google Scholar 

  21. Sheng, W., Shan, P., Chen, S., Liu, Y., Alsaadi, F.E.: A niching evolutionary algorithm with adaptive negative correlation learning for neural network ensemble. Neurocomputing 247, 173–182 (2017)

    Article  Google Scholar 

  22. Kottathra, K., Attikiouzel, Y.: A novel multicriteria optimization algorithm for the structure determination of multilayer feedforward neural networks. J. Netw. Comput. Appl. 19(2), 135–147 (1996)

    Article  Google Scholar 

  23. Kupinski, M.A., Anastasio, M.A.: Multiobjective genetic optimization of diagnostic classifiers with implications for generating receiver operating characteristic curves. IEEE Trans. Med. Imaging 18(8), 675–685 (1999)

    Article  Google Scholar 

  24. Chandra, A., Yao, X.: Ensemble learning using multi-objective evolutionary algorithms. J. Math. Model. Algorithms 5(4), 417–445 (2006). https://doi.org/10.1007/s10852-005-9020-3

    Article  MathSciNet  MATH  Google Scholar 

  25. Thompson, S.: Pruning boosted classifiers with a real valued genetic algorithm. In: Miles, R., Moulton, M., Bramer, M. (eds.) Research and Development in Expert Systems XV, pp. 133–146. Springer, London (1999). https://doi.org/10.1007/978-1-4471-0835-1_9

    Chapter  Google Scholar 

  26. Zhou, Z.-H., Tang, W.: Selective ensemble of decision trees. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 476–483. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-39205-X_81

    Chapter  Google Scholar 

  27. Mao, W., Tian, M., Cao, X., Xu, J.: Model selection of extreme learning machine based on multi-objective optimization. Neural Comput. Appl. 22(3–4), 521–529 (2013). https://doi.org/10.1007/s00521-011-0804-2

    Article  Google Scholar 

  28. Pavelski, L.M., Delgado, M.R., Almeida, C.P., Gonçalves, R.A., Venske, S.M.: Extreme learning surrogate models in multi-objective optimization based on decomposition. Neurocomputing 180, 55–67 (2016)

    Article  Google Scholar 

  29. Deb, K.: Multi-objective genetic algorithms: problem difficulties and construction of test problems. Evol. Comput. 7(3), 205–230 (1999)

    Article  Google Scholar 

  30. Liang, J., Yue, C., Qu, B.: Multimodal multi-objective optimization: a preliminary study. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 2454–2461. IEEE (2016)

    Google Scholar 

  31. Yue, C., Qu, B., Liang, J.: A multiobjective particle swarm optimizer using ring topology for solving multimodal multiobjective problems. IEEE Trans. Evol. Comput. 22(5), 805–817 (2017)

    Article  Google Scholar 

  32. Liang, J., Guo, Q., Yue, C., Qu, B., Yu, K.: A self-organizing multi-objective particle swarm optimization algorithm for multimodal multi-objective problems. In: Tan, Y., Shi, Y., Tang, Q. (eds.) ICSI 2018. LNCS, vol. 10941, pp. 550–560. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93815-8_52

    Chapter  Google Scholar 

  33. Yue, C., Qu, B., Yu, K., Liang, J., Li, X.: A novel scalable test problem suite for multimodal multiobjective optimization. Swarm Evol. Comput. 48, 62–71 (2019)

    Article  Google Scholar 

  34. Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml

  35. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61976237,61922072, 61876169, 61673404).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University, Zhengzhou, China

    Jing Liang, Panpan Wei, Kunjie Yu, Caitong Yue, Yi Hu & Shilei Ge

  2. School of Electronic and Information Engineering, Zhongyuan University of Technology, Zhengzhou, China

    Boyang Qu

Authors
  1. Jing Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panpan Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boyang Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kunjie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caitong Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yi Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shilei Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panpan Wei .

Editor information

Editors and Affiliations

  1. Huazhong University of Science and Technology, Wuhan, China

    Linqiang Pan

  2. Zhengzhou University, Zhengzhou, China

    Jing Liang

  3. Zhongyuan University of Technology, Zhengzhou, China

    Boyang Qu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liang, J. et al. (2020). Ensemble Learning Based on Multimodal Multiobjective Optimization. In: Pan, L., Liang, J., Qu, B. (eds) Bio-inspired Computing: Theories and Applications. BIC-TA 2019. Communications in Computer and Information Science, vol 1159. Springer, Singapore. https://doi.org/10.1007/978-981-15-3425-6_24

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-3425-6_24

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-3424-9

  • Online ISBN: 978-981-15-3425-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation