Skip to main content

Advertisement

SpringerLink
Log in

Adaptive memetic method of multi-objective genetic evolutionary algorithm for backpropagation neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, multi-objective evolutionary optimization algorithms have shown success in different areas of research. Due to their efficiency and power, many researchers have concentrated on adapting evolutionary algorithms to generate Pareto solutions. This paper proposes a new memetic adaptive multi-objective evolutionary algorithm that is based on a three-term backpropagation network (MAMOT). This algorithm is an automatic search method for optimizing the parameters and performance of neural networks, and it relies on the use of the adaptive non-dominated sorting genetic algorithm-II integrated with the backpropagation algorithm, being used as a local search method. The presented MAMOT employs a self-adaptive mechanism toward improving the performance of the proposed algorithm and a local optimizer improving all the individuals in a population in order to obtain better accuracy and connection weights. In addition, it selects an appropriate number of hidden nodes simultaneously. The proposed method was applied to 11 datasets representing pattern classification problems, including two-class, multi-class and complex data reflecting real problems. Experiments were performed, and the results indicated that the proposed method is viable in pattern classification tasks compared to a multi-objective genetic algorithm based on a three-term backpropagation network (MOGAT) and some of the methods mentioned in the literature. The statistical analysis results of the t test and Wilcoxon signed-ranks test also show that the performance of the proposed method is significantly better than MOGAT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Salari N, Shohaimi S, Najafi F, Nallappan M, Karishnarajah I (2014) A novel hybrid classification model of genetic algorithms, modified k-nearest neighbor and developed backpropagation neural network. PLoS One 9:e112987

    Article  Google Scholar 

  2. Prieto A, Bellas F, Duro RJ, Lopez-Peña F (2007) Auto adjustable ANN-based classification system for optimal high dimensional data analysis. Computational and ambient intelligence. Springer, Berlin, pp 588–596

    Google Scholar 

  3. Garro BA, Sossa H, Vázquez RA (2011) Evolving neural networks: a comparison between differential evolution and particle swarm optimization. Springer, Berlin, pp 447–454

    Google Scholar 

  4. Qasem SN, Shamsuddin SM, Zain AM (2011) Multi-objective hybrid evolutionary algorithms for radial basis function neural network design. Knowl Based Syst 27:475–497

    Article  Google Scholar 

  5. Sagar G, Chalam SV, Singh MK (2011) Evolutionary algorithm for optimal connection weights in artificial neural networks. Int J Eng (IJE) 5:333

    Google Scholar 

  6. Yu Q, Peng J (2011) Music category based on adaptive mutation particle swarm optimization BP neural network. In: Wu Y (ed) Advances in computer, communication, control and automation, vol 121. Lecture notes in electrical engineering. Berlin, Heidelberg, pp 657–663

    Chapter  Google Scholar 

  7. Abraham A, Nath B (2000) Optimal design of neural nets using hybrid algorithms. In: PRICAI 2000 topics in artificial intelligence, Springer, Berlin, pp 510–520

    Chapter  Google Scholar 

  8. Zhang R, Tao J (2017) Data-driven modeling using improved multi-objective optimization based neural network for coke furnace system. IEEE Trans Ind Electron 64:3147–3155

    Article  Google Scholar 

  9. Ahmad F, Isa NAM, Hussain Z, Osman MK, Sulaiman SN (2015) A GA-based feature selection and parameter optimization of an ANN in diagnosing breast cancer. Pattern Anal Appl 18:861–870

    Article  MathSciNet  Google Scholar 

  10. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87:1423–1447

    Article  Google Scholar 

  11. Hernández C, Schütze O, Sun J-Q (2017) Global multi-objective optimization by means of cell mapping techniques. In: EVOLVE–a bridge between probability, set oriented numerics and evolutionary computation VII, Springer, Cham, pp 25–56

    Chapter  Google Scholar 

  12. Zhou A, Qu B-Y, Li H, Zhao S-Z, Suganthan PN et al (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evolut Comput 1:32–49

    Article  Google Scholar 

  13. Cruz-Ramírez M, Hervás-Martínez C, Jurado-Expósito M, López-Granados F (2012) A multi-objective neural network based method for cover crop identification from remote sensed data. Expert Syst Appl 39:10038–10048

    Article  Google Scholar 

  14. Ibrahim AO, Shamsuddin SM, Ahmad NB, Qasem SN (2013) Three-term backpropagation network based on elitist multiobjective genetic algorithm for medical diseases diagnosis classification. Life Sci J 10(4):1815–1822

    Google Scholar 

  15. Ak R, Li Y, Vitelli V, Zio E, Droguett EL et al (2013) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Exp Syst Appl 40:1205–1212

    Article  Google Scholar 

  16. Fernández JC, Cruz-Ramírez M, Hervás-Martínez C (2018) Sensitivity versus accuracy in ensemble models of artificial neural networks from multi-objective evolutionary algorithms. Neural Comput Appl 30(1):289–305

    Article  Google Scholar 

  17. Abbass HA (2003) Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Comput 15:2705–2726

    Article  MATH  Google Scholar 

  18. Fernandez Caballero JC, Martínez FJ, Hervás C, Gutiérrez PA (2010) Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks. IEEE Trans Neural Netw 21:750–770

    Article  Google Scholar 

  19. Silva VV, Fleming PJ, Sugimoto J, Yokoyama R (2008) Multiobjective optimization using variable complexity modelling for control system design. Appl Soft Comput 8:392–401

    Article  Google Scholar 

  20. Gorzałczany MB, Rudziński F (2017) Interpretable and accurate medical data classification—a multi-objective genetic-fuzzy optimization approach. Exp Syst Appl 71:26–39

    Article  Google Scholar 

  21. Pettersson F, Chakraborti N, Saxén H (2007) A genetic algorithms based multi-objective neural net applied to noisy blast furnace data. Appl Soft Comput 7:387–397

    Article  Google Scholar 

  22. Jin Y, Sendhoff B, Körner E (2005) Evolutionary multi-objective optimization for simultaneous generation of signal-type and symbol-type representations. Springer, Berlin, pp 752–766

    MATH  Google Scholar 

  23. Jin Y, Sendhoff B (2008) Pareto-based multiobjective machine learning: an overview and case studies. IEEE Trans Syst Man Cybern Part C Appl Rev 38:397–415

    Google Scholar 

  24. Garcıa-Pedrajas N, Ortiz-Boyer D, Hervás-Martınez C (2004) Cooperative coevolution of generalized multi-layer perceptrons. Neurocomputing 56:257–283

    Article  Google Scholar 

  25. Delgado M, Cuéllar MP, Pegalajar MC (2008) Multiobjective hybrid optimization and training of recurrent neural networks. IEEE Trans Syst Man Cybern Part B Cybern 38:381–403

    Article  Google Scholar 

  26. Jin Y, Sendhoff B, Körner E (2006) Simultaneous generation of accurate and interpretable neural network classifiers. Multi-objective machine learning. Springer, Berlin, pp 291–312

    Google Scholar 

  27. Wiegand S, Igel C, Handmann U (2004) Evolutionary multi-objective optimisation of neural networks for face detection. Int J Comput Intell Appl 4:237–253

    Article  Google Scholar 

  28. Abbass HA (2002) An evolutionary artificial neural networks approach for breast cancer diagnosis. Artif Intell Med 25:265–281

    Article  Google Scholar 

  29. Almeida LM, Ludermir TB (2010) A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks. Neurocomputing 73:1438–1450

    Article  Google Scholar 

  30. Cruz-Ramírez M, Sánchez-Monedero J, Fernández-Navarro F, Fernández J, Hervás-Martínez C (2010) Memetic pareto differential evolutionary artificial neural networks to determine growth multi-classes in predictive microbiology. Evol Intel 3:187–199

    Article  Google Scholar 

  31. Qasem SN, Shamsuddin SM (2011) Memetic elitist pareto differential evolution algorithm based radial basis function networks for classification problems. Appl Soft Comput 11:5565–5581

    Article  Google Scholar 

  32. Cruz-Ramírez M, Hervás-Martínez C, Fernández JC, Briceño J, de la Mata M (2012) Multi-objective evolutionary algorithm for donor-recipient decision system in liver transplants. Eur J Oper Res 222(2):317–327

    Article  Google Scholar 

  33. Abbass HA (2002) The self-adaptive pareto differential evolution algorithm. In: Proceedings of the 2002 congress on evolutionary computation. pp 831–836

  34. Huang VL, Zhao SZ, Mallipeddi R, Suganthan PN (2009) Multi-objective optimization using self-adaptive differential evolution algorithm. In: IEEE congress on evolutionary computation pp 190–194

  35. Zeng F, Low MYH, Decraene J, Zhou S, Cai W (2010) Self-adaptive mechanism for multi-objective evolutionary algorithms. In Proceedings of the international multiconference of engineers and computer scientists, pp 7–12

  36. Shim VA, Tan KC, Tang H (2014) Adaptive memetic computing for evolutionary multiobjective optimization. IEEE Trans Cybern 45(4):610–621

    Article  Google Scholar 

  37. Caruana R, Niculescu-Mizil A (2004) Data mining in metric space: an empirical analysis of supervised learning performance criteria. In: ACM. pp 69–78

  38. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874

    Article  Google Scholar 

  39. Thangaraj R, Pant M, Abraham A, Badr Y (2009) Hybrid evolutionary algorithm for solving global optimization problems. Hybrid artificial intelligence systems. Springer, Berlin, pp 310–318

    Book  Google Scholar 

  40. Krasnogor N, Smith J (2005) A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans Evolut Comput 9:474–488

    Article  Google Scholar 

  41. Fernández J, Hervás C, Martínez F, Gutiérrez P, Cruz M (2009) Memetic Pareto differential evolution for designing artificial neural networks in multiclassification problems using cross-entropy versus sensitivity. In: Hybrid artificial intelligence systems. pp 433–441

    Google Scholar 

  42. Schaffer JD (1985) Multiple objective optimization with vector evaluated genetic algorithms. L. Erlbaum Associates Inc, Mahwah, pp 93–100

    Google Scholar 

  43. Srinivas N, Deb K (1994) Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evol Comput 2:221–248

    Article  Google Scholar 

  44. Fonseca CM, Fleming PJ (1998) Multiobjective optimization and multiple constraint handling with evolutionary algorithms. II. Application example. IEEE Trans Syst Man Cybern Part A Syst Hum 28:38–47

    Article  Google Scholar 

  45. Zweiri Y, Whidborne J, Seneviratne L (2003) A three-term backpropagation algorithm. Neurocomputing 50:305–318

    Article  MATH  Google Scholar 

  46. Zweiri YH (2007) Optimization of a three-term backpropagation algorithm used for neural network learning. Int J Comput Intell 3:322–327

    Google Scholar 

  47. Deb K (1999) An introduction to genetic algorithms. Indian Acad Sci 24:293–315

    MathSciNet  MATH  Google Scholar 

  48. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6:182–197

    Article  Google Scholar 

  49. Ak R, Li Y, Vitelli V, Zio E, Droguett EL et al (2012) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Exp Syst Appl 40(4):1205–1212

    Article  Google Scholar 

  50. Ramesh S, Kannan S, Baskar S (2012) Application of modified NSGA-II algorithm to multi-objective reactive power planning. Appl Soft Comput 12(2):741–753

    Article  Google Scholar 

  51. Deb K, Sindhya K, Okabe T. Self-adaptive simulated binary crossover for real-parameter optimization; 2007. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2007), UCL London. pp 1187–1194

  52. Zhang C, Ren M, Zhang B (2013) A self-adaptive multi-objective genetic algorithm for the QoS based routing and wavelength allocation problem in WDM network. Opt Int J Light Electron Opt 124(20):4571–4575

    Article  Google Scholar 

  53. Lara A, Sanchez G, Coello Coello CA, Schutze O (2010) HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans Evolut Comput 14:112–132

    Article  Google Scholar 

  54. Qasem SN, Shamsuddin SM, Hashi SZM, Darus M, Al-Shammari E (2013) Memetic multiobjective particle swarm optimization-based radial basis function network for classification problems. Inf Sci 239:165–190

    Article  MathSciNet  Google Scholar 

  55. Asuncion A, Newman DJ (2007) UCI machine learning repository. http://www.icsuciedu/$\sim$mlearn/{MLR}epositoryhtml. Accessed 18 July 2017

  56. Hervás C, Silva M, Gutiérrez PA, Serrano A (2008) Multilogistic regression by evolutionary neural network as a classification tool to discriminate highly overlapping signals: qualitative investigation of volatile organic compounds in polluted waters by using headspace-mass spectrometric analysis. Chemom Intell Lab Syst 92(2):179–185

    Article  Google Scholar 

  57. Goh C-K, Teoh E-J, Tan KC (2008) Hybrid multiobjective evolutionary design for artificial neural networks. IEEE Trans Neural Netw 19:1531–1548

    Article  Google Scholar 

  58. Bhuvana J, Aravindan C (2016) Memetic algorithm with Preferential Local Search using adaptive weights for multi-objective optimization problems. Soft Comput 20(4):1365–1388

    Article  Google Scholar 

  59. Stehlík M, Střelec L, Thulin M (2014) On robust testing for normality in chemometrics. Chemom Intell Lab Syst 130:98–108

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and the editor for their useful advice and constructive comments.

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, Alzaiem Alazhari University, 13311, Khartoum North, Sudan

    Ashraf Osman Ibrahim

  2. Arab Open University, Khartoum, Sudan

    Ashraf Osman Ibrahim

  3. Faculty of computer science, Future University, Khartoum, Sudan

    Ashraf Osman Ibrahim

  4. UTM Big Data Centre, Universiti Teknologi Malaysia, 81310, Skudai, Johor, Malaysia

    Siti Mariyam Shamsuddin

  5. Machine Intelligence Research Labs (MIR Labs), Scientific Network for Innovation and Research Excellence, Auburn, Washington, USA

    Ajith Abraham

  6. Computer Science Department, College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia

    Sultan Noman Qasem

  7. Computer Science Department, Faculty of Applied Sciences, Taiz University, Taiz, Yemen

    Sultan Noman Qasem

Authors
  1. Ashraf Osman Ibrahim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siti Mariyam Shamsuddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ajith Abraham
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sultan Noman Qasem
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashraf Osman Ibrahim.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ibrahim, A.O., Shamsuddin, S.M., Abraham, A. et al. Adaptive memetic method of multi-objective genetic evolutionary algorithm for backpropagation neural network. Neural Comput & Applic 31, 4945–4962 (2019). https://doi.org/10.1007/s00521-018-03990-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-03990-0

Keywords

Search

Navigation