Skip to main content
SpringerLink
Log in

Memetic cooperative coevolution of Elman recurrent neural networks

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Cooperative coevolution decomposes an optimisation problem into subcomponents and collectively solves them using evolutionary algorithms. Memetic algorithms provides enhancement to evolutionary algorithms with local search. Recently, the incorporation of local search into a memetic cooperative coevolution method has shown to be efficient for training feedforward networks on pattern classification problems. This paper applies the memetic cooperative coevolution method for training recurrent neural networks on grammatical inference problems. The results show that the proposed method achieves better performance in terms of optimisation time and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Acampora G, Cadenas J, Loia V, Ballester E (2011) Achieving memetic adaptability by means of agent-based machine learning. IEEE Trans Indus Inform 7(4):557–569

    Article  Google Scholar 

  • Acampora G, Gaeta M, Loia V (2011) Combining multi-agent paradigm and memetic computing for personalized and adaptive learning experiences. Comput Intell 27(2):141–165

    Article  MathSciNet  Google Scholar 

  • Acampora G, Loia V, Salerno S, Vitiello A (2012) A hybrid evolutionary approach for solving the ontology alignment problem. Int J Intell Syst 27(3):189–216

    Article  Google Scholar 

  • Chandra R, Frean M, Zhang M (2010) An encoding scheme for cooperative coevolutionary neural networks. In: 23rd Australian joint conference on artificial intelligence. Lecture notes in artificial intelligence. Springer, Adelaide, Australia, in Press

  • Chandra R, Frean M, Zhang M (2011a) A memetic framework for cooperative coevolution of recurrent neural networks. In: The 2011 international joint conference on neural networks (IJCNN), pp 673–680

  • Chandra R, Frean M, Zhang M (2011b) Modularity adaptation in cooperative coevolution of feedforward neural networks. In: The 2011 international joint conference on neural networks (IJCNN), pp 681–688

  • Chandra R, Frean M, Zhang M, Omlin CW (2011c) Encoding subcomponents in cooperative co-evolutionary recurrent neural networks. Neurocomputing 74(17):3223–3234

    Article  Google Scholar 

  • Chandra R, Frean M, Zhang M (2012a) Adapting modularity during learning in cooperative co-evolutionary recurrent neural networks. Soft Comput Fusion Found Methodol Appl 16(6):1009–1020

    Google Scholar 

  • Chandra R, Frean M, Zhang M (2012b) Crossover-based local search in cooperative co-evolutionary feedforward neural networks. Appl Soft Comput 12(9):2924–2932

    Article  Google Scholar 

  • Chandra R, Frean M, Zhang M (2012c) On the issue of separability for problem decomposition in cooperative neuro-evolution. Neurocomputing 87:33–40

    Article  Google Scholar 

  • Deb K, Anand A, Joshi D (2002) A computationally efficient evolutionary algorithm for real-parameter optimization. Evol Comput 10(4):371–395

    Article  Google Scholar 

  • Elman JL (1990) Finding structure in time. Cogn Sci 14:179–211

    Article  Google Scholar 

  • Giles CL, Horne BG, Lin T (1995) Learning a class of large finite state machines with a recurrent neural network. Neural Netw 8(9):1359–1365

    Article  Google Scholar 

  • Glover FW, Kochenberger GA (2003) Handbook of metaheuristics. Springer, Berlin

    MATH  Google Scholar 

  • Gomez F, Mikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3–4):317–342

    Article  Google Scholar 

  • Gomez F, Schmidhuber J, Miikkulainen R (2008) Accelerated neural evolution through cooperatively coevolved synapses. J Mach Learn Res 9:937–965

    MATH  MathSciNet  Google Scholar 

  • Gomez FJ (2003) Robust non-linear control through neuroevolution. Technical Report AI-TR-03-303, PhD thesis, Department of Computer Science, The University of Texas at Austin

  • Haykin S, Principe J, Sejnowski T, McWhirter J (2006) New directions in statistical signal processing: from systems to brain. MIT Press, Cambridge

    Google Scholar 

  • Kazarlis SA, Papadakis SE, Theocharis IB, Petridis V (2001) Microgenetic algorithms as generalized hill-climbing operators for ga optimization. IEEE Trans Evolut Comput 5(3):204–217

    Article  Google Scholar 

  • Kolen J, Kremer S (2001) A field guide to dynamical recurrent networks. IEEE Press, Piscataway, NJ, USA

    Google Scholar 

  • Kremer S (1995) On the computational power of elman-style recurrent networks. IEEE Trans Neural Netw 6(4):1000–1004

    Article  Google Scholar 

  • Lin C-J, Chen C-H, Lin C-T (January 2009) A hybrid of cooperative particle swarm optimization and cultural algorithm for neural fuzzy networks and its prediction applications. Trans Syst Man Cyber Part C 39:55–68

  • Lozano M, Herrera F, Krasnogor N, Molina D (2004) Real-coded memetic algorithms with crossover hill-climbing. Evol Comput 12:273–302

    Google Scholar 

  • Lozano M, Herrera F, Krasnogor N, Molina D (2004) Real-coded memetic algorithms with crossover hill-climbing. Evol Comput 12(3):273–302

    Article  Google Scholar 

  • Manolios P, Fanelli R (1994) First-order recurrent neural networks and deterministic finite state automata. Neural Comput 6(6):1155–1173

    Article  Google Scholar 

  • Medsker L, Jain L (1999) Recurrent neural networks: design and application, computer intelligence. CRC Press, Florida, USA

    Book  Google Scholar 

  • Molina D, Lozano M, Garca-Martnez C, Herrera F (2010) Memetic algorithms for continuous optimisation based on local search chains. Evol Comput 18(1):27–63

    Article  Google Scholar 

  • Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. Tech Rep

  • Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. Technical Report 826, Caltech Concurrent Computation Program

  • Moscato P (2003) A gentle introduction to memetic algorithms. In: Handbook of Metaheuristics. Kluwer Academic Publishers, Dordrecht, pp 105–144

  • Nguyen QH, Ong Y-S, Lim MH (2009) A probabilistic memetic framework. IEEE Trans Evolut Comput 13(3):604–623

    Article  Google Scholar 

  • Ong YS, Keane A (2004) Meta-lamarckian learning in memetic algorithms. IEEE Trans Evolut Comput 8(2):99–110

    Article  Google Scholar 

  • Pham DT, Karaboga D (1999) Training elman and jordan networks for system identification using genetic algorithms. Artif Intell Eng 13(2):107–117

    Article  Google Scholar 

  • Potter MA, De Jong KA (2000) Cooperative coevolution: An architecture for evolving coadapted subcomponents. Evol Comput 8(1):1–29

    Article  Google Scholar 

  • Potter MA, Jong KAD (1994) A cooperative coevolutionary approach to function optimization. Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature. Springer, London, UK, PPSN III, pp 249–257

    Google Scholar 

  • Pošik P (2009) Bbob-benchmarking the generalized generation gap model with parent centric crossover. In: Proceedings of the 11th annual conference companion on genetic and evolutionary computation conference: late breaking papers. GECCO ’09, pp 2321–2328

  • Robinson T (1994) An application of recurrent nets to phone probability estimation. IEEE Trans Neural Netw 5:298–305

    Article  Google Scholar 

  • Seyab RA, Cao Y (2008) Nonlinear system identification for predictive control using continuous time recurrent neural networks and automatic differentiation. J Process Control 18(6):568–581

    Article  Google Scholar 

  • Smith J (2007) Coevolving memetic algorithms: a review and progress report. IEEE Trans Syst Man Cybern Part B Cybern 37(1):6–17

    Article  Google Scholar 

  • Tang J, Lim MH, Ong YS (2007) Diversity-adaptive parallel memetic algorithm for solving large scale combinatorial optimization problems. Soft Comput 11(9):873–888

    Article  Google Scholar 

  • Tomita M (1982) Dynamic construction of finite automata from examples using hill-climbing. Proceedings of the fourth annual cognitive science Conference. MI, Ann Arbor, pp 105–108

  • Watts DJ (1999) Small worlds: The dynamics of networks between order and randomness. Princeton University Press, Princeton

    Google Scholar 

  • Zhou Z, Ong YS, Lim MH, Lee BS (2007) Memetic algorithm using multi-surrogates for computationally expensive optimization problems. Soft Comput 11(10):957–971

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Science Technology and Environment, School of Computing, Information and Mathematical Sciences, University of the South Pacific, Suva, Fiji

    Rohitash Chandra

Authors
  1. Rohitash Chandra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rohitash Chandra.

Additional information

Communicated by G. Acampora.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chandra, R. Memetic cooperative coevolution of Elman recurrent neural networks. Soft Comput 18, 1549–1559 (2014). https://doi.org/10.1007/s00500-013-1160-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-013-1160-1

Keywords

Search

Navigation