Skip to main content

Cuckoo Search and Bat Algorithm Applied to Training Feed-Forward Neural Networks

  • Chapter
  • First Online:
Recent Advances in Swarm Intelligence and Evolutionary Computation

Part of the book series: Studies in Computational Intelligence ((SCI,volume 585))

  • 2036 Accesses

Abstract

Training of feed-forward neural networks is a well-known and important hard optimization problem, frequently used for classification purpose. Swarm intelligence metaheuristics have been successfully used for such optimization problems. In this chapter we present how cuckoo search and bat algorithm, as well as the modified version of the bat algorithm, were adjusted and applied to the training of feed-forward neural networks. We used these three algorithms to search for the optimal synaptic weights of the neural network in order to minimize the function errors. The testing was done on four well-known benchmark classification problems. Since the number of neurons in hidden layers may strongly influence the performance of artificial neural networks, we considered several neural networks architectures for different number of neurons in the hidden layers. Results show that the performance of the cuckoo search and bat algorithms is comparable to other state-of-the-art nondeterministic optimization algorithms, with some advantage of the cuckoo search. However, modified bat algorithm outperformed all other algorithms which shows great potential of this recent swarm intelligence algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Du, K.L.: Clustering: a neural network approach. Neural Netw. 23(1), 89–107 (2010)

    Article  Google Scholar 

  2. Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)

    Article  Google Scholar 

  3. Kim, T.: Pattern recognition using artificial neural network: a review. Inf. Secur. Assur. Commun. Comput. Inf. Sci. 76, 138–148 (2010)

    Google Scholar 

  4. Shrivastava, G., Karmakar, S., Kowar, M.K., Guhathakurta, P.: Application of artificial neural networks in weather forecasting: a comprehensive literature review. Int. J. Comput. Appl. 51(18), 17–29 (2012)

    Article  Google Scholar 

  5. Perez, M.: Artificial neural networks and bankruptcy forecasting: a state of the art. Neural Comput. Appl. 15(2), 154–163 (2006)

    Article  Google Scholar 

  6. Haykin, S.: Neural Networks and Learning Machines. Prentice Hall, New Jersey (2009)

    Google Scholar 

  7. Marquardt, D.W.: An algorithm for least squares estimation of non-linear parameters. J. Soc. Ind. Appl. Math. 11(2), 431–441 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  8. Jovanovic, R., Tuba, M.: Ant colony optimization algorithm with pheromone correction strategy for the minimum connected dominating set problem. Comput. Sci. Inf. Syst.(ComSIS) 10(1), 133–149 (2013)

    Article  Google Scholar 

  9. Tuba, M., Jovanovic, R.: Improved ACO algorithm with pheromone correction strategy for the traveling salesman problem. Int. J. Comput. Commun. Control 8(3), 477–485 (2013)

    Article  MathSciNet  Google Scholar 

  10. Jovanovic, R., Tuba, M.: An ant colony optimization algorithm with improved pheromone correction strategy for the minimum weight vertex cover problem. Appl. Soft Comput. 11(8), 5360–5366 (2011)

    Article  Google Scholar 

  11. Jovanovic, R., Tuba, M.: An analysis of different variations of ant colony optimization to the minimum weight vertex cover problem. WSEAS Trans. Inf. Sci. Appl. 6(6), 936–945 (2009)

    Google Scholar 

  12. Bacanin, N., Tuba, M.: Artificial bee colony (ABC) algorithm for constrained optimization improved with genetic operators. Stud. Inf. Control 21(2), 137–146 (2012)

    Google Scholar 

  13. Brajevic, I., Tuba, M.: An upgraded artificial bee colony algorithm (ABC) for constrained optimization problems. J. Intell. Manuf. 24(4), 729–740 (2013)

    Article  Google Scholar 

  14. Subotic, M., Tuba, M.: Parallelized multiple swarm artificial bee colony algorithm (MS-ABC) for global optimization. Stud. Inf. Control 23(1), 117–126 (2014)

    Google Scholar 

  15. Tuba, M., Bacanin, N.: Artificial bee colony algorithm hybridized with firefly metaheuristic for cardinality constrained mean-variance portfolio problem. Appl. Math. Inf. Sci. 8(6), 2831–2844 (2014)

    Article  MathSciNet  Google Scholar 

  16. Tuba, M., Brajevic, I., Jovanovic, R.: Hybrid seeker optimization algorithm for global optimization. Appl. Math. Inf. Sci. 7(3), 867–875 (2013)

    Article  MathSciNet  Google Scholar 

  17. Tuba, M., Bacanin, N.: Improved seeker optimization algorithm hybridized with firefly algorithm for constrained optimization problems. Neurocomputing 143, 197–207 (2014). doi:10.1016/j.neucom.2014.06.006

    Article  Google Scholar 

  18. Yang, X.S.: Firefly algorithms for multimodal optimization. Stochastic Algorithms: Found. Appl. LNCS 5792, 169–178 (2009)

    Article  Google Scholar 

  19. Fister, I., Fister, I.J., Yang, X.S., Brest, J.: A comprehensive review of firefly algorithms. Swarm Evol. Comput. 13(1), 34–46 (2013)

    Article  Google Scholar 

  20. Yang, X.S.: Multiobjective firefly algorithm for continuous optimization. Eng. Comput. 29(2), 175–184 (2013)

    Article  Google Scholar 

  21. Tuba, M., Bacanin, N.: Upgraded firefly algorithm for portfolio optimization problem. In: Proceedings of the 16th IEEE International Conference on Computer Modelling and Simulation, UKSim-AMSS 2014, pp. 112–117. IEEE, New Jersey (2014)

    Google Scholar 

  22. Tuba, M., Bacanin, N.: JPEG quantization tables selection by the firefly algorithm. In: Proceedings of the 4th IEEE International Conference on Multimedia Computing and Systems (ICMCS14), IEEE Catalog Number: CFP14050-CDR, Submission 402, pp. 153–158. IEEE, New Jersey (2014)

    Google Scholar 

  23. Bacanin, N., Tuba, M.: Firefly algorithm for cardinality constrained mean-variance portfolio optimization problem with entropy diversity constraint. Sci. World J. 2014 (721521) 16 (2014). doi:10.1155/2014/721521

  24. Che, Z.G., Chiang, T.A., Che, Z.H.: Feed-forward neural networks training: A comparison between genetic algorithm and back-propagation learning algorithm. Int. J. Innov. Comput. Inf. Control 7(10), 5839–5850 (2011)

    Google Scholar 

  25. Mendes, R., Cortez, P., Rocha, M., Neves, J.: Particle swarm for feedforward neural network training. In: Proceedings of the International Joint Conference on Neural Networks 2, 1895–1899 (2002)

    Google Scholar 

  26. Ilonen, J., Kamarainen, J.K., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Process. Lett. 17(1), 93–105 (2003)

    Article  Google Scholar 

  27. Karaboga, D., Akay, B., Ozturk, C.: Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. Lecture Notes in Computer Science: Modeling Decisions for Artificial Intelligence 4617, 318–329 (2007)

    Google Scholar 

  28. Karaboga, D., Ozturk, C.: Neural networks training by artificial bee colony algorithm on pattern classification. Neural Netw. World 19(3), 279–292 (2009)

    Google Scholar 

  29. Brajevic, I., Tuba, M.: Training feed-forward neural networks using firefly algorithm. In: Proceedings of the 12th International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases (AIKED ’13), pp. 156–161 (2013)

    Google Scholar 

  30. Yang, X.S., Deb, S.: Engineering optimization by cuckoo search. Int. J. Math. Model. Numer. Optim. 1(4), 330–343 (2010)

    MATH  Google Scholar 

  31. Brajevic, I., Tuba, M.: Cuckoo search and firefly algorithm applied to multilevel image thresholding. In: X.S. Yang (ed.) Cuckoo Search and Firefly Algorithm: Theory and Applications, Studies in Computational Intelligence, vol. 516, pp. 115–139. Springer International Publishing, Switzerland (2014)

    Google Scholar 

  32. Yang, X.S., Deb, S.: Cuckoo search via Levy flights. In: Proceedings of World Congress on Nature & Biologically Inspired Computing (NaBIC 2009), pp. 210–214 (2009)

    Google Scholar 

  33. Gandomi, A.H., Yang, X.S., Alavi, A.H.: Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng. Comput. 29(1), 17–35 (2013)

    Article  MathSciNet  Google Scholar 

  34. Zhou, Y., Zheng, H., Luo, Q., Wu, J.: An improved cuckoo search algorithm for solving planar graph coloring problem. Appl. Math. Inf. Sci. 7(2), 785–792 (2013)

    Article  MathSciNet  Google Scholar 

  35. Yang, X.S.: A new metaheurisitic bat-inspired algorithm. Stud. Comput. Intell. 284, 65–74 (2010)

    Article  Google Scholar 

  36. Yang, X.S.: Bat algorithm for multi-objective optimisation. Int. J. Bio-Inspired Comput. 3(5), 267–274 (2011)

    Google Scholar 

  37. Yang, X.S., Gandomi, A.H.: Bat algorithm: a novel approach for global engineering optimization. Eng. Comput. 29(5), 464–483 (2012)

    Article  Google Scholar 

  38. Moustafa, A.A., Alqadi, Z.A., Shahroury, E.A.: Performance evaluation of artificial neural networks for spatial data analysis. WSEAS Trans. Comput. 10(4), 115–124 (2011)

    Google Scholar 

  39. Tuba, M., Subotic, M., Stanarevic, N.: Modified cuckoo search algorithm for unconstrained optimization problems. In: Proceedings of the European Computing Conference (ECC 2011) pp. 263–268 (2011)

    Google Scholar 

  40. Tuba, M., Subotic, M., Stanarevic, N.: Performance of a modified cuckoo search algorithm for unconstrained optimization problems. WSEAS Trans. Syst. 11(2), 62–74 (2012)

    Google Scholar 

  41. Layeb, A.: A novel quantum-inspired cuckoo search for knapsack problems. Int. J. Bio-Inspired Comput. 3(5), 297–305 (2011)

    Google Scholar 

  42. Bacanin, N.: Implementation and performance of an object-oriented software system for cuckoo search algorithm. Int. J. Math. Comput. Simul. 6(1), 185–193 (2012)

    Google Scholar 

  43. Huang, G.Q., Zhao, W.J., Lu, Q.Q.: Bat algorithm with global convergence for solving large-scale optimization problem. Appl. Res. Comput. 30(3), 1–10 (2013)

    Article  Google Scholar 

  44. Du, Z.Y., Liu, B.: Image matching using a bat algorithm with mutation. Appl. Mech. Mater. 203(1), 88–93 (2012)

    Google Scholar 

  45. Tsai, P.W., Pan, J.S., Liao, B.Y., Tsai, M.J., Istanda, V.: Bat algorithm inspired algorithm for solving numerical optimization problems. Appl. Mech. Mater. 148–149, 134–137 (2011)

    Article  Google Scholar 

  46. Alihodzic, A., Tuba, M.: Improved hybridized bat algorithm for global numerical optimization. In: Proceedings of the 16th IEEE International Conference on Computer Modelling and Simulation, UKSim-AMSS 2014, pp. 57–62 (2014)

    Google Scholar 

  47. Alihodzic, A., Tuba, M.: Improved bat algorithm applied to multilevel image thresholding. Sci World J 2014(176718), 16 (2014). doi:10.1155/2014/176718

  48. Battiti, R.: First- and second-order methods for learning: Between steepest descent and newtons method. Neural Comput. 4(2), 141–166 (1992)

    Article  Google Scholar 

  49. Srivastava, P.R., Varshney, A., Nama, P., Yang, X.S.: Software test effort estimation: a model based on cuckoo search. Int. J. Bio-Inspired Comput. 4(5), 278–285 (2012)

    Article  Google Scholar 

  50. Yang, X.S., Deb, S.: Cuckoo search: recent advances and applications. Neural Comput. Appl. 24(1, SI), 169–174 (2014). doi:10.1007/s00521-013-1367-1

  51. Yang, X.S., He, X.: Bat algorithm: literature review and applications. Int. J. Bio-Inspired Comput. 5(3), 141–149 (2013). doi:10.1504/IJBIC.2013.055093

    Article  Google Scholar 

  52. He, X., Ding, W.J., Yang, X.S.: Bat algorithm based on simulated annealing and gaussian perturbations. Neural Comput. Appl. 25(2), 459–468 (2014)

    Article  Google Scholar 

  53. Wang, G., Guo, L.: A novel hybrid bat algorithm with harmony search for global numerical optimization. J. Appl. Math. 2013, 1–22 (2013)

    MATH  Google Scholar 

  54. Liu, D., Hohil, M.E., Smith, S.H.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48(1–4), 477–488 (2002)

    Article  MATH  Google Scholar 

  55. Newman, D., Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998). http://www.ics.uci.edu/mlearn/MLRepository.html

Download references

Acknowledgments

This reserach was supported by Ministry of Education and Science of Republic of Serbia, Grant III-44006.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Milan Tuba .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Tuba, M., Alihodzic, A., Bacanin, N. (2015). Cuckoo Search and Bat Algorithm Applied to Training Feed-Forward Neural Networks. In: Yang, XS. (eds) Recent Advances in Swarm Intelligence and Evolutionary Computation. Studies in Computational Intelligence, vol 585. Springer, Cham. https://doi.org/10.1007/978-3-319-13826-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13826-8_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13825-1

  • Online ISBN: 978-3-319-13826-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics