Skip to main content
SpringerLink
Log in

Training Neural Networks by Continuation Particle Swarm Optimization

  • Conference paper
  • First Online:
Progress in Artificial Intelligence and Pattern Recognition (IWAIPR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11047))

Abstract

Artificial Neural Networks research field is among the areas of major activity in Artificial Intelligence. Conventional training approaches applied to neural networks present several theoretical and computational limitations. In this paper we propose an approach for Artificial Neural Network training based on optimization by continuation and Particle Swarm Optimization algorithm. The objective is to reduce overall execution time of training without causing negative effects in accuracy. Our proposal is compared with Standard Particle Swarm Optimization algorithm using public benchmark datasets. Experimental results show that the optimization by continuation approach reduces execution time required to perform training in about \(20\%{-}50\%\) without statistically significant loss of accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmed, H.M., Youssef, B.A., Elkorany, A.S., Saleeb, A.A., El-Samie, F.A.: Hybrid gray wolf optimizer-artificial neural network classification approach for magnetic resonance brain images. Appl. Opt. 57(7), B25–B31 (2018)

    Article  Google Scholar 

  2. Bonyadi, M.R., Michalewicz, Z.: Particle swarm optimization for single objective continuous space problems: a review (2017)

    Google Scholar 

  3. Chang, P., Yang, J., Yang, J., You, T.: Respiratory signals prediction based on particle swarm optimization and back propagation neural networks. In: AIP Conference Proceedings, vol. 1834, p. 040049. AIP Publishing (2017)

    Google Scholar 

  4. Chatterjee, S., Dey, N., Ashour, A.S., Drugarin, C.V.A.: Electrical energy output prediction using cuckoo search based artificial neural network. In: Yang, X.-S., Nagar, A.K., Joshi, A. (eds.) Smart Trends in Systems, Security and Sustainability. LNNS, vol. 18, pp. 277–285. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-6916-1_26

    Chapter  Google Scholar 

  5. Chen, W., Wang, X.A., Zhang, W., Xu, C.: Phishing detection research based on PSO-BP neural network. In: Barolli, L., Xhafa, F., Javaid, N., Spaho, E., Kolici, V. (eds.) EIDWT 2018. LNDECT, pp. 990–998. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-319-75928-9_91

    Chapter  Google Scholar 

  6. Chilimbi, T., Suzue, Y., Apacible, J., Kalyanaraman, K.: Project Adam: building an efficient and scalable deep learning training system. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 571–582 (2014)

    Google Scholar 

  7. Cortez, P., Cerdeira, A., Almeida, F., Matos, T., Reis, J.: Modeling wine preferences by data mining from physicochemical properties. Decis. Support Syst. 47(4), 547–553 (2009)

    Article  Google Scholar 

  8. Dauphin, Y.N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., Bengio, Y.: Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2933–2941. Curran Associates, Inc. (2014)

    Google Scholar 

  9. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  10. Griewank, A.: On automatic differentiation. Math. Program.: Recent Dev. Appl. 6(6), 83–107 (1989)

    MathSciNet  MATH  Google Scholar 

  11. Hashem, M., Hassanein, A.S.: Jaw fracture classification using meta heuristic firefly algorithm with multi-layered associative neural networks. Clust. Comput. 1–8 (2018)

    Google Scholar 

  12. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. LION 5, 507–523 (2011)

    Google Scholar 

  13. Janzamin, M., Sedghi, H., Anandkumar, A.: Beating the perils of non-convexity: guaranteed training of neural networks using tensor methods. CoRR abs/1506.08473 (2015)

    Google Scholar 

  14. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  15. Li, K., Mao, S., Li, X., Wu, Z., Meng, H.: Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks. Speech Commun. 96, 28–36 (2018)

    Article  Google Scholar 

  16. Livni, R., Shalev-Shwartz, S., Shamir, O.: On the computational efficiency of training neural networks. In: Advances in Neural Information Processing Systems, pp. 855–863 (2014)

    Google Scholar 

  17. Mobahi, H., Fisher III, J.W.: A theoretical analysis of optimization by Gaussian continuation. In: AAAI, pp. 1205–1211 (2015)

    Google Scholar 

  18. Nawi, N.M., Rehman, M., Khan, A.: Ws-bp: an efficient wolf search based back-propagation algorithm. In: International Conference on Mathematics, Engineering And Industrial Applications 2014 (ICoMEIA 2014). vol. 1660, p. 050027. AIP Publishing (2015)

    Google Scholar 

  19. Nayak, J., Naik, B., Behera, H.: A novel nature inspired firefly algorithm with higher order neural network: performance analysis. Eng. Sci. Technol. Int. J. 19, 197–211 (2015)

    Article  Google Scholar 

  20. Neeraja, D., Swaroop, G.: Prediction of compressive strength of concrete using artificial neural networks. Res. J. Pharm. Technol. 10(1), 35–40 (2017)

    Article  Google Scholar 

  21. Owaki, S., Fukumoto, Y., Sakamoto, T., Yamamoto, N., Nakamura, M.: Experimental demonstration of SPM compensation based on digital signal processing using a three-layer neural-network for 40-Gbit/s optical 16QAM signal. IEICE Commun. Express 7(1), 13–18 (2018)

    Article  Google Scholar 

  22. Rashid, M., Kamal, K., Zafar, T., Sheikh, Z., Shah, A., Mathavan, S.: Energy prediction of a combined cycle power plant using a particle swarm optimization trained feedforward neural network. In: 2015 International Conference on Mechanical Engineering, Automation and Control Systems (MEACS), pp. 1–5. IEEE (2015)

    Google Scholar 

  23. Sreeshakthy, M., Preethi, J.: Classification of human emotion from deap EEG signal using hybrid improved neural networks with cuckoo search. BRAIN. Broad Res. Artif. Intell. Neurosci. 6(3–4), 60–73 (2016)

    Google Scholar 

  24. Tibshirani, R.J., Tibshirani, R.: A bias correction for the minimum error rate in cross-validation. Ann. Appl. Stat. 3, 822–829 (2009)

    Article  MathSciNet  Google Scholar 

  25. Yu, M., Yang, P., Wei, S.: Railway obstacle detection algorithm using neural network. In: AIP Conference Proceedings. vol. 1967, p. 040017. AIP Publishing (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidad de las Ciencias Informáticas, Havana, Cuba

    Jairo Rojas-Delgado & Rafael Trujillo-Rasúa

Authors
  1. Jairo Rojas-Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael Trujillo-Rasúa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jairo Rojas-Delgado .

Editor information

Editors and Affiliations

  1. Universidad de las Ciencias Informáticas, Havana, Cuba

    Yanio Hernández Heredia

  2. Universidad de las Ciencias Informáticas, Havana, Cuba

    Vladimir Milián Núñez

  3. Universidad de las Ciencias Informáticas, Havana, Cuba

    José Ruiz Shulcloper

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rojas-Delgado, J., Trujillo-Rasúa, R. (2018). Training Neural Networks by Continuation Particle Swarm Optimization. In: Hernández Heredia, Y., Milián Núñez, V., Ruiz Shulcloper, J. (eds) Progress in Artificial Intelligence and Pattern Recognition. IWAIPR 2018. Lecture Notes in Computer Science(), vol 11047. Springer, Cham. https://doi.org/10.1007/978-3-030-01132-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01132-1_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01131-4

  • Online ISBN: 978-3-030-01132-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation