Abstract
Training of feed-forward neural networks is a well-known and important hard optimization problem, frequently used for classification purpose. Swarm intelligence metaheuristics have been successfully used for such optimization problems. In this chapter we present how cuckoo search and bat algorithm, as well as the modified version of the bat algorithm, were adjusted and applied to the training of feed-forward neural networks. We used these three algorithms to search for the optimal synaptic weights of the neural network in order to minimize the function errors. The testing was done on four well-known benchmark classification problems. Since the number of neurons in hidden layers may strongly influence the performance of artificial neural networks, we considered several neural networks architectures for different number of neurons in the hidden layers. Results show that the performance of the cuckoo search and bat algorithms is comparable to other state-of-the-art nondeterministic optimization algorithms, with some advantage of the cuckoo search. However, modified bat algorithm outperformed all other algorithms which shows great potential of this recent swarm intelligence algorithm.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Du, K.L.: Clustering: a neural network approach. Neural Netw. 23(1), 89–107 (2010)
Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)
Kim, T.: Pattern recognition using artificial neural network: a review. Inf. Secur. Assur. Commun. Comput. Inf. Sci. 76, 138–148 (2010)
Shrivastava, G., Karmakar, S., Kowar, M.K., Guhathakurta, P.: Application of artificial neural networks in weather forecasting: a comprehensive literature review. Int. J. Comput. Appl. 51(18), 17–29 (2012)
Perez, M.: Artificial neural networks and bankruptcy forecasting: a state of the art. Neural Comput. Appl. 15(2), 154–163 (2006)
Haykin, S.: Neural Networks and Learning Machines. Prentice Hall, New Jersey (2009)
Marquardt, D.W.: An algorithm for least squares estimation of non-linear parameters. J. Soc. Ind. Appl. Math. 11(2), 431–441 (1963)
Jovanovic, R., Tuba, M.: Ant colony optimization algorithm with pheromone correction strategy for the minimum connected dominating set problem. Comput. Sci. Inf. Syst.(ComSIS) 10(1), 133–149 (2013)
Tuba, M., Jovanovic, R.: Improved ACO algorithm with pheromone correction strategy for the traveling salesman problem. Int. J. Comput. Commun. Control 8(3), 477–485 (2013)
Jovanovic, R., Tuba, M.: An ant colony optimization algorithm with improved pheromone correction strategy for the minimum weight vertex cover problem. Appl. Soft Comput. 11(8), 5360–5366 (2011)
Jovanovic, R., Tuba, M.: An analysis of different variations of ant colony optimization to the minimum weight vertex cover problem. WSEAS Trans. Inf. Sci. Appl. 6(6), 936–945 (2009)
Bacanin, N., Tuba, M.: Artificial bee colony (ABC) algorithm for constrained optimization improved with genetic operators. Stud. Inf. Control 21(2), 137–146 (2012)
Brajevic, I., Tuba, M.: An upgraded artificial bee colony algorithm (ABC) for constrained optimization problems. J. Intell. Manuf. 24(4), 729–740 (2013)
Subotic, M., Tuba, M.: Parallelized multiple swarm artificial bee colony algorithm (MS-ABC) for global optimization. Stud. Inf. Control 23(1), 117–126 (2014)
Tuba, M., Bacanin, N.: Artificial bee colony algorithm hybridized with firefly metaheuristic for cardinality constrained mean-variance portfolio problem. Appl. Math. Inf. Sci. 8(6), 2831–2844 (2014)
Tuba, M., Brajevic, I., Jovanovic, R.: Hybrid seeker optimization algorithm for global optimization. Appl. Math. Inf. Sci. 7(3), 867–875 (2013)
Tuba, M., Bacanin, N.: Improved seeker optimization algorithm hybridized with firefly algorithm for constrained optimization problems. Neurocomputing 143, 197–207 (2014). doi:10.1016/j.neucom.2014.06.006
Yang, X.S.: Firefly algorithms for multimodal optimization. Stochastic Algorithms: Found. Appl. LNCS 5792, 169–178 (2009)
Fister, I., Fister, I.J., Yang, X.S., Brest, J.: A comprehensive review of firefly algorithms. Swarm Evol. Comput. 13(1), 34–46 (2013)
Yang, X.S.: Multiobjective firefly algorithm for continuous optimization. Eng. Comput. 29(2), 175–184 (2013)
Tuba, M., Bacanin, N.: Upgraded firefly algorithm for portfolio optimization problem. In: Proceedings of the 16th IEEE International Conference on Computer Modelling and Simulation, UKSim-AMSS 2014, pp. 112–117. IEEE, New Jersey (2014)
Tuba, M., Bacanin, N.: JPEG quantization tables selection by the firefly algorithm. In: Proceedings of the 4th IEEE International Conference on Multimedia Computing and Systems (ICMCS14), IEEE Catalog Number: CFP14050-CDR, Submission 402, pp. 153–158. IEEE, New Jersey (2014)
Bacanin, N., Tuba, M.: Firefly algorithm for cardinality constrained mean-variance portfolio optimization problem with entropy diversity constraint. Sci. World J. 2014 (721521) 16 (2014). doi:10.1155/2014/721521
Che, Z.G., Chiang, T.A., Che, Z.H.: Feed-forward neural networks training: A comparison between genetic algorithm and back-propagation learning algorithm. Int. J. Innov. Comput. Inf. Control 7(10), 5839–5850 (2011)
Mendes, R., Cortez, P., Rocha, M., Neves, J.: Particle swarm for feedforward neural network training. In: Proceedings of the International Joint Conference on Neural Networks 2, 1895–1899 (2002)
Ilonen, J., Kamarainen, J.K., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Process. Lett. 17(1), 93–105 (2003)
Karaboga, D., Akay, B., Ozturk, C.: Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. Lecture Notes in Computer Science: Modeling Decisions for Artificial Intelligence 4617, 318–329 (2007)
Karaboga, D., Ozturk, C.: Neural networks training by artificial bee colony algorithm on pattern classification. Neural Netw. World 19(3), 279–292 (2009)
Brajevic, I., Tuba, M.: Training feed-forward neural networks using firefly algorithm. In: Proceedings of the 12th International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases (AIKED ’13), pp. 156–161 (2013)
Yang, X.S., Deb, S.: Engineering optimization by cuckoo search. Int. J. Math. Model. Numer. Optim. 1(4), 330–343 (2010)
Brajevic, I., Tuba, M.: Cuckoo search and firefly algorithm applied to multilevel image thresholding. In: X.S. Yang (ed.) Cuckoo Search and Firefly Algorithm: Theory and Applications, Studies in Computational Intelligence, vol. 516, pp. 115–139. Springer International Publishing, Switzerland (2014)
Yang, X.S., Deb, S.: Cuckoo search via Levy flights. In: Proceedings of World Congress on Nature & Biologically Inspired Computing (NaBIC 2009), pp. 210–214 (2009)
Gandomi, A.H., Yang, X.S., Alavi, A.H.: Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng. Comput. 29(1), 17–35 (2013)
Zhou, Y., Zheng, H., Luo, Q., Wu, J.: An improved cuckoo search algorithm for solving planar graph coloring problem. Appl. Math. Inf. Sci. 7(2), 785–792 (2013)
Yang, X.S.: A new metaheurisitic bat-inspired algorithm. Stud. Comput. Intell. 284, 65–74 (2010)
Yang, X.S.: Bat algorithm for multi-objective optimisation. Int. J. Bio-Inspired Comput. 3(5), 267–274 (2011)
Yang, X.S., Gandomi, A.H.: Bat algorithm: a novel approach for global engineering optimization. Eng. Comput. 29(5), 464–483 (2012)
Moustafa, A.A., Alqadi, Z.A., Shahroury, E.A.: Performance evaluation of artificial neural networks for spatial data analysis. WSEAS Trans. Comput. 10(4), 115–124 (2011)
Tuba, M., Subotic, M., Stanarevic, N.: Modified cuckoo search algorithm for unconstrained optimization problems. In: Proceedings of the European Computing Conference (ECC 2011) pp. 263–268 (2011)
Tuba, M., Subotic, M., Stanarevic, N.: Performance of a modified cuckoo search algorithm for unconstrained optimization problems. WSEAS Trans. Syst. 11(2), 62–74 (2012)
Layeb, A.: A novel quantum-inspired cuckoo search for knapsack problems. Int. J. Bio-Inspired Comput. 3(5), 297–305 (2011)
Bacanin, N.: Implementation and performance of an object-oriented software system for cuckoo search algorithm. Int. J. Math. Comput. Simul. 6(1), 185–193 (2012)
Huang, G.Q., Zhao, W.J., Lu, Q.Q.: Bat algorithm with global convergence for solving large-scale optimization problem. Appl. Res. Comput. 30(3), 1–10 (2013)
Du, Z.Y., Liu, B.: Image matching using a bat algorithm with mutation. Appl. Mech. Mater. 203(1), 88–93 (2012)
Tsai, P.W., Pan, J.S., Liao, B.Y., Tsai, M.J., Istanda, V.: Bat algorithm inspired algorithm for solving numerical optimization problems. Appl. Mech. Mater. 148–149, 134–137 (2011)
Alihodzic, A., Tuba, M.: Improved hybridized bat algorithm for global numerical optimization. In: Proceedings of the 16th IEEE International Conference on Computer Modelling and Simulation, UKSim-AMSS 2014, pp. 57–62 (2014)
Alihodzic, A., Tuba, M.: Improved bat algorithm applied to multilevel image thresholding. Sci World J 2014(176718), 16 (2014). doi:10.1155/2014/176718
Battiti, R.: First- and second-order methods for learning: Between steepest descent and newtons method. Neural Comput. 4(2), 141–166 (1992)
Srivastava, P.R., Varshney, A., Nama, P., Yang, X.S.: Software test effort estimation: a model based on cuckoo search. Int. J. Bio-Inspired Comput. 4(5), 278–285 (2012)
Yang, X.S., Deb, S.: Cuckoo search: recent advances and applications. Neural Comput. Appl. 24(1, SI), 169–174 (2014). doi:10.1007/s00521-013-1367-1
Yang, X.S., He, X.: Bat algorithm: literature review and applications. Int. J. Bio-Inspired Comput. 5(3), 141–149 (2013). doi:10.1504/IJBIC.2013.055093
He, X., Ding, W.J., Yang, X.S.: Bat algorithm based on simulated annealing and gaussian perturbations. Neural Comput. Appl. 25(2), 459–468 (2014)
Wang, G., Guo, L.: A novel hybrid bat algorithm with harmony search for global numerical optimization. J. Appl. Math. 2013, 1–22 (2013)
Liu, D., Hohil, M.E., Smith, S.H.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48(1–4), 477–488 (2002)
Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998). http://www.ics.uci.edu/mlearn/MLRepository.html
Acknowledgments
This reserach was supported by Ministry of Education and Science of Republic of Serbia, Grant III-44006.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Tuba, M., Alihodzic, A., Bacanin, N. (2015). Cuckoo Search and Bat Algorithm Applied to Training Feed-Forward Neural Networks. In: Yang, XS. (eds) Recent Advances in Swarm Intelligence and Evolutionary Computation. Studies in Computational Intelligence, vol 585. Springer, Cham. https://doi.org/10.1007/978-3-319-13826-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-13826-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13825-1
Online ISBN: 978-3-319-13826-8
eBook Packages: EngineeringEngineering (R0)