Abstract
This paper proposes a method named Population-based Algorithm (PBA) to decide the best hyperparameters for a neural network (NN). The study focuses on which type of hyperparameters achieve better results in neural network problems. Population-based algorithm inspired from evolutionary algorithms and uses basic steps of genetic algorithms. The distinctive feature of our algorithm from genetic algorithms is fitness evaluation of individuals. To test our approach, we implemented our algorithm to a handwritten digits recognition problem to find the best hyperparameters for a simple neural network and we reached 98.66 accuracy score. Finally, we conclude, how PBA used in neural networks for the best way.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fiszelew, A., et al.: Finding optimal neural network architecture using genetic algorithms. Adv. Comput. Sci. Eng. Res. Comput. Sci. 27, 15–24 (2007)
Bergstra, J.S., et al.: Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems (2011)
Vose, A., et al.: Deep neural network hyperparameter optimization with genetic algorithms (2017)
Fridrich, M.: Hyperparameter optimization of artificial neural network in customer churn prediction using genetic algorithm. Trendy Ekon. Manag. 11(28), 9–21 (2017)
Bergstra, J., Bengio, Y.: Random search for hyperparameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)
LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST handwritten digits database. http://yann.lecun.com/exdb/mnist
Patil, S.S, Bhalchandra, A.S: pattern recognition using genetic algorithm (2017)
Valdez, F., Melin, P., Parra, H.: Parallel genetic algorithms for optimization of modular neural networks in pattern recognition (2011)
Jaderberg, M., et al.: Population based training of neural networks. arXiv:1711.09846 (2017)
Suganuma, M., Shirakawa, S., Nagao, T.: A genetic programming approach to designing convolutional neural network architectures. In: Proceedings of the Genetic and Evolutionary Computation Conference. ACM (2017)
Dufourq, E., Bassett, B.A.: EDEN: evolutionary deep networks for efficient machine learning. In: Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech, 2017). IEEE (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Nalçakan, Y., Ensari, T. (2019). Decision of Neural Networks Hyperparameters with a Population-Based Algorithm. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R., Sciacca, V. (eds) Machine Learning, Optimization, and Data Science. LOD 2018. Lecture Notes in Computer Science(), vol 11331. Springer, Cham. https://doi.org/10.1007/978-3-030-13709-0_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-13709-0_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-13708-3
Online ISBN: 978-3-030-13709-0
eBook Packages: Computer ScienceComputer Science (R0)