Skip to main content
SpringerLink
Log in

Model Selection in Committees of Evolved Convolutional Neural Networks Using Genetic Algorithms

  • Conference paper
  • First Online:
Book cover Intelligent Data Engineering and Automated Learning – IDEAL 2018 (IDEAL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11314))

Abstract

Neuroevolution is a technique that has been successfully applied for over three decades in order to optimize certain aspects of neural networks by applying evolutionary algorithms. However, only in the recent years, the increase of computational resources has enabled to apply such techniques to deep and convolutional neural networks, where the number of hyperparameters is significantly large.

In recent years, deep and convolutional neural networks are outperforming classical machine learning for many different tasks, including computer vision, natural language processing, signal processing, activity recognition, etc. In this context, neuroevolution can be useful since there are no analytic approaches for determining optimal network architectures or hyperparameters, therefore attaining better performance. Moreover, in some cases, committees (also called ensembles) are used, which combine two or more models in order to improve results even more. Neuroevolution can be of particular importance in this case, since evolutionary algorithms evolve a whole population of individuals, making it easy to build ensembles out of models chosen among those in the evolved population.

In this paper, we explore the application of genetic algorithms for carrying out model selection in a context of neuroevolution. Thus, the best models will be selected from a population of evolved individuals in order to maximize an overall objective function. This approach is tested using the well-known MNIST database as benchmark, and it obtains results which are highly competitive when compared with the state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akhtyamova, L., Ignatov, A., Cardiff, J.: A Large-scale CNN ensemble for medication safety analysis. In: Frasincar, F., Ittoo, A., Nguyen, L.M., Métais, E. (eds.) NLDB 2017. LNCS, vol. 10260, pp. 247–253. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59569-6_29

    Chapter  Google Scholar 

  2. Baker, B., Gupta, O., Naik, N., Raskar, R.: Designing neural network architectures using reinforcement learning. In: Proceedings of the 5th International Conference on the Learning Repr. (2017)

    Google Scholar 

  3. Baldominos, A., Saez, Y., Isasi, P.: Evolutionary convolutional neural networks: an application to handwriting recognition. Neurocomputing 283, 38–52 (2018)

    Article  Google Scholar 

  4. Baldominos, A., Saez, Y., Isasi, P.: Evolutionary design of convolutional neural networks for human activity recognition in sensor-rich environments. Sensors 18(4), 1288 (2018)

    Article  Google Scholar 

  5. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  6. Chang, J.R., Chen, Y.S.: Batch-normalized maxout network in network. arXiv arXiv:1511.02583 (2015)

  7. Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: Proceedings of the 2011 International Conference on Document Analysis and Recognition, pp. 1135–1139 (2011)

    Google Scholar 

  8. Davison, J.: DEvol: Automated deep neural network design via genetic programming (2017). https://github.com/joeddav/devol. Accessed 01 July 2017

  9. Desell, T.: Large scale evolution of convolutional neural networks using volunteer computing. In: Proceedings 2017 Genetic Evolutionary Computation Conference Companion, pp. 127–128 (2017)

    Google Scholar 

  10. Fernando, C., et al.: Convolution by evolution: differentiable pattern producing networks. In: Proceedings 2016 Genetic Evolutionary Computation Conference, pp. 109–116 (2016)

    Google Scholar 

  11. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    Article  Google Scholar 

  12. Hinton, G., Srivastava, N., Swersky, K.: RMSProp: divide the gradient by a running average of its recent magnitude. Neural networks for machine learning, Coursera lecture 6e (2012)

    Google Scholar 

  13. Kassahun, Y., Sommer, G.: Efficient reinforcement learning through evolutionary acquisition of neural topologies. In: Proceedings of the 2005 European Symposium on Artificial Neural Networks, pp. 259–266 (2005)

    Google Scholar 

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv arXiv:1412.6980 (2014)

  15. Koutník, J., Schmidhuber, J., Gomez, F.: Evolving deep unsupervised convolutional networks for vision-based reinforcement learning. In: Proceedings 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 541–548 (2014)

    Google Scholar 

  16. Loshchilov, I., Hutter, F.: CMA-ES for hyperparameter optimization of deep neural networks. arXiv abs/1604.07269 (2016)

    Google Scholar 

  17. Miikkulainen, R., et al.: Evolving deep neural networks. arXiv abs/1703.00548 (2017)

    Google Scholar 

  18. Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)

    Article  Google Scholar 

  19. Real, E., et al.: Large-scale evolution of image classifiers. arXiv abs/1703.01041 (2017)

    Google Scholar 

  20. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  21. Suganuma, M., Shirakawa, S., Nagao, T.: A genetic programming approach to designing convolutional neural network architectures. In: Proceedings 2017 Genetic Genetic and Evolutionary Computation Conference, pp. 497–504 (2017)

    Google Scholar 

  22. Sultana, J., Rani, M.U., Farquad, M.: An extensive survey on some deep learning applications. In: Proceedings of the 2018 IADS International Conference on Computing, Communications & Data Engineering (2018)

    Google Scholar 

  23. Verbancsics, P., Harguess, J.: Image classification using generative neuroevolution for deep learning. In: Proceedings 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 488–493 (2015)

    Google Scholar 

  24. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 13 (2018)

    Google Scholar 

  25. Xie, L., Yuille, A.: Genetic CNN. arXiv abs/1703.01513 (2017)

    Google Scholar 

  26. Yao, X., Liu, Y.: A new evolutionary system for evolving artificial neural networks. IEEE Trans. Neural Netw. 8(3), 694–713 (1997)

    Article  Google Scholar 

  27. Young, S.R., Rose, D.C., Karnowsky, T.P., Lim, S.H., Patton, R.M.: Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: Proceedings Workshop on Machine Learning in High-Performance Computing Environments (2015)

    Google Scholar 

  28. Zhang, L., Wang, S., Liu, B.: Deep learning for sentiment analysis: a survey. Interdiscip. Rev. Data Min. Knowl. Discov. 8, e1253 (2018)

    Article  Google Scholar 

  29. Zhang, Q., Yang, L.T., Chen, Z., Li, P.: A survey on deep learning for big data. Inf. Fusion 42, 146–157 (2018)

    Article  Google Scholar 

Download references

Acknowledgement

This research is supported by the Spanish Ministry of Education, Culture and Sport through FPU fellowship with identifier FPU13/03917.

Author information

Authors and Affiliations

  1. Computer Science Department, Universidad Carlos III de Madrid, Avenida de la Universidad 30, 28911, Leganes, Spain

    Alejandro Baldominos, Yago Saez & Pedro Isasi

Authors
  1. Alejandro Baldominos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yago Saez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro Isasi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandro Baldominos .

Editor information

Editors and Affiliations

  1. University of Manchester, Manchester, UK

    Hujun Yin

  2. Autonomous University of Madrid, Madrid, Spain

    David Camacho

  3. Campus of Gualtar, University of Minho, Braga, Portugal

    Paulo Novais

  4. University of Seville, Seville, Spain

    Antonio J. Tallón-Ballesteros

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baldominos, A., Saez, Y., Isasi, P. (2018). Model Selection in Committees of Evolved Convolutional Neural Networks Using Genetic Algorithms. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science(), vol 11314. Springer, Cham. https://doi.org/10.1007/978-3-030-03493-1_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03493-1_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03492-4

  • Online ISBN: 978-3-030-03493-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation