Skip to main content
Springer Nature Link
Log in

Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations

  • Conference paper
Evolutionary Multi-Criterion Optimization (EMO 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3410))

Included in the following conference series:

Abstract

It has been a controversial issue in the research of cognitive science and artificial intelligence whether signal-type representations (typically connectionist networks) or symbol-type representations (e.g., semantic networks, production systems) should be used. Meanwhile, it has also been recognized that both types of information representations might exist in the human brain. In addition, symbol-type representations are often very helpful in gaining insights into unknown systems. For these reasons, comprehensible symbolic rules need to be extracted from trained neural networks. In this paper, an evolutionary multi-objective algorithm is employed to generate multiple models that facilitate the generation of signal-type and symbol-type representations simultaneously. It is argued that one main difference between signal-type and symbol-type representations lies in the fact that the signal-type representations are models of a higher complexity (fine representation), whereas symbol-type representations are models of a lower complexity (coarse representation). Thus, by generating models with a spectrum of model complexity, we are able to obtain a population of models of both signal-type and symbol-type quality, although certain post-processing is needed to get a fully symbol-type representation. An illustrative example is given on generating neural networks for the breast cancer diagnosis benchmark problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Abbass, H.A.: An evolutionary artificialneuralnet works approach for breast cancer diagnosis. Artificial Intelligence in Medicine 25(3), 265–281 (2002)

    Article  Google Scholar 

  2. Abbass, H.A.: Speeding up back-propagation using multiobjective evolutionary algorithms. Neural Computation 15(11), 2705–2726 (2003)

    Article  MATH  Google Scholar 

  3. Andrews, R., Diederich, J., Tickle, A.: A survey and critique of techniques for extracting rules from trained artificialneuralnet works. Knowledge Based Systems 8(6), 373–389 (1995)

    Article  Google Scholar 

  4. Badre, D., Wagner, A.: Semantic retrieval, mnemonic control, and prefrontal cortex. Behavior and Cognitive Neuroscience Reviews 1(3), 206–218 (2002)

    Article  Google Scholar 

  5. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)

    Google Scholar 

  6. Burnham, K.P., Anderson, D.R.: Model Selection and Multimodel Inference, 2nd edn. Springer, New York (2002)

    MATH  Google Scholar 

  7. Coello Coello, C., Veldhuizen, D., Lamont, G.: Evolutionary algorithms for solving multi-objective problems. Kluwer Academic, New York (2002)

    MATH  Google Scholar 

  8. Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Chichester (2001)

    MATH  Google Scholar 

  9. Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In: Parallel Problem Solving from Nature, vol. VI, pp. 849–858 (2000)

    Google Scholar 

  10. Duch, W., Adamczak, R., Grabczewski, K.: Extraction of logical rules from backpropagation networks. Neural Processing Letters 7, 1–9 (1998)

    Article  Google Scholar 

  11. Duch, W., Setiono, R., Zurada, J.: Computational intelligence methods for rule-based data understanding. Proceedings of the IEEE 92(5), 771–805 (2004)

    Article  Google Scholar 

  12. Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: A criticalanalysis. Cognition 28(3), 3–71 (1988)

    Article  Google Scholar 

  13. Gabrieli, J., Poldrack, R., Desmond, J.: The role of left prefrontal cortex in langrange and memory. Proceedings of the national Academy of Sciences 95, 906–913 (1998)

    Article  Google Scholar 

  14. Hüsken, M., Gayko, J.E., Sendhoff, B.: Optimization for problem classes –Neural networks that learn to learn. In: Yao, X., Fogel, D.B. (eds.) IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (ECNN 2000), pp. 98–109. IEEE Press, Los Alamitos (2000)

    Chapter  Google Scholar 

  15. Igeland, C., Hüsken, M.: Improving the Rprop learning algorithm. In: Proceedings of the 2nd ICSC International Symposium on Neural Computation, pp. 115–121 (2000)

    Google Scholar 

  16. Ishibuchi, H., Yamamoto, T.: Evolutionary multiobjective optimization for generating an ensemble of fuzzy rule-based classifiers. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1077–1088 (2003)

    Google Scholar 

  17. Ishikawa, M.: Rule extraction by successive regularization. Neural Networks 13, 1171–1183 (2000)

    Article  Google Scholar 

  18. Jin, Y.: Advanced Fuzzy Systems Design and Applications. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  19. Jin, Y., Okabe, T., Sendhoff, B.: Neuralnet work regularization and ensembling using multi-objective evolutionary algorithms. In: Congress on Evolutionary Computation, pp. 1–8. IEEE, Los Alamitos (2004)

    Google Scholar 

  20. Jin, Y., Sendhoff, B.: Extracting interpretable fuzzy rules from RBF networks. Neural Processing Letters 17(2), 149–164 (2003)

    Article  MATH  Google Scholar 

  21. Martin, A., Chao, L.: Semantic memory and the brain: Structure and process. Current Opinions in Neurobiology 11, 194–201 (2001)

    Article  Google Scholar 

  22. Miller, D.A., Zurada, J.M.: A dynamical system perspective of structural learning with forgetting. IEEE Transactions on Neural Networks 9(3), 508–515 (1998)

    Article  Google Scholar 

  23. Prechelt, L.: PROBEN1 - a set of neuralnet work benchmark problems and benchmarking rules. Technical Report 21/94, Fakultát für Informatik, Universität Karlsruhe (1994)

    Google Scholar 

  24. Quinlan, J.R.: C4.5 Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)

    Google Scholar 

  25. Reed, R.D., Marks II, R.J.: Neural Smithing. MIT Press, Cambridge (1999)

    Google Scholar 

  26. Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropgation learning: The RPROP algorithm. In: IEEE international Conference on Neural Networks, vol. 1, pp. 586–591. IEEE, New York (1993)

    Chapter  Google Scholar 

  27. Setiono, R.: Generating concise and accurate classification rules for breast cancer disgnosis. Artificial Intelligence in Medicine 18, 205–219 (2000)

    Article  Google Scholar 

  28. Setiono, R., Liu, H.: Symbolic representation of neural networks. IEEE Computer 29(3), 71–77 (1996)

    Google Scholar 

  29. Taha, I., Ghosh, J.: Symbolic interpretation of artificialneuralnet works. IEEE Transactions on Knowledge and Data Engineering 11(3), 448–463 (1999)

    Article  Google Scholar 

  30. de Roselito Teixeira, A., Braga, A.P., Takahashi, R.H.C., Saldanha, R.R.: Improving generalization of MLPs with multi-objective optimization. Neurocomputing 35, 189–194 (2000)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Honda Research Institute Europe, 63073, Offenbach/Main, Germany

    Yaochu Jin, Bernhard Sendhoff & Edgar Körner

Authors
  1. Yaochu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Sendhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edgar Körner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Departamento de Computación, CINVESTAV-IPN (Evolutionary Computation Group), 07300, México, D.F., México

    Carlos A. Coello Coello

  2. Centro de Investigación en Matemáticas (CIMAT), A.P. 402, Guanajuato, 36000, Mexico

    Arturo Hernández Aguirre

  3. Computer Engineering and Networks Laboratory, ETH, Zurich, Switzerland

    Eckart Zitzler

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Jin, Y., Sendhoff, B., Körner, E. (2005). Evolutionary Multi-objective Optimization for Simultaneous Generation of Signal-Type and Symbol-Type Representations. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds) Evolutionary Multi-Criterion Optimization. EMO 2005. Lecture Notes in Computer Science, vol 3410. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-31880-4_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-31880-4_52

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-24983-2

  • Online ISBN: 978-3-540-31880-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics