Skip to main content
SpringerLink
Log in
Book cover

Constructive Neural Networks pp 49–70Cite as

Constructive Neural Network Algorithms That Solve Highly Non-separable Problems

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 258))

Abstract

Learning from data with complex non-local relations and multimodal class distribution is still very hard for standard classification algorithms. Even if an accurate solution is found the resulting model may be too complex for a given data and will not generalize well. New types of learning algorithms are needed to extend capabilities of machine learning systems to handle such data. Projection pursuit methods can avoid “curse of dimensionality” by discovering interesting structures in low-dimensional subspace. This paper introduces constructive neural architectures based on projection pursuit techniques that are able to discover simplest models of data with inherent highly complex logical structures. The key principle is to look for transformations that discover interesting structures, going beyond error functions and separability.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Asuncion, A., Newman, D.: UCI repository of machine learning databases (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html

  2. Bengio, Y., Delalleau, O., Roux, N.L.: The curse of dimensionality for local kernel machines. Technical Report 1258, Dṕartement d’informatique et recherche opérationnelle, Université de Montréal (2005)

    Google Scholar 

  3. Duch, W.: K-separability. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 188–197. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  4. Duch, W.: Towards comprehensive foundations of computational intelligence. In: Duch, W., Mandziuk, J. (eds.) Challenges for Computational Intelligence, vol. 63, pp. 261–316. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  5. Duch, W., Adamczak, R., Gra̧bczewski, K.: A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks 12, 277–306 (2001)

    Article  Google Scholar 

  6. Duch, W., Grudziński, K.: Meta-learning via search combined with parameter optimization. In: Rutkowski, L., Kacprzyk, J. (eds.) Advances in Soft Computing, pp. 13–22. Physica Verlag, Springer, New York (2002)

    Google Scholar 

  7. Duch, W., Jankowski, N.: Survey of neural transfer functions. Neural Computing Surveys 2, 163–213 (1999)

    Google Scholar 

  8. Duch, W., Jankowski, N.: Taxonomy of neural transfer functions. In: International Joint Conference on Neural Networks, Como, Italy, vol. III, pp. 477–484. IEEE Press, Los Alamitos (2000)

    Google Scholar 

  9. Duch, W., Jankowski, N.: Transfer functions: hidden possibilities for better neural networks. In: 9th European Symposium on Artificial Neural Networks, Brusells, Belgium, pp. 81–94. De-facto publications (2001)

    Google Scholar 

  10. Friedman, J.H.: Exploratory projection pursuit. Journal of the American Statistical Association 82, 249–266 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  11. Friedman, J.H., Tukey, J.W.: A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. 23(9), 881–890 (1974)

    Article  MATH  Google Scholar 

  12. Grochowski, M., Duch, W.: A Comparison of Methods for Learning of Highly Non-separable Problems. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 566–577. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  13. Grochowski, M., Jankowski, N.: Comparison of instance selection algorithms II. Results and comments. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 580–585. Springer, Heidelberg (2004)

    Google Scholar 

  14. Haykin, S.: Neural Networks - A Comprehensive Foundation. Maxwell MacMillian Int., New York (1994)

    MATH  Google Scholar 

  15. Huber, P.J.: Projection pursuit. Annals of Statistics 13, 435–475 (1985), http://www.stat.rutgers.edu/~rebecka/Stat687/huber.pdf

    Article  MATH  MathSciNet  Google Scholar 

  16. Iyoda, E.M., Nobuhara, H., Hirota, K.: A solution for the n-bit parity problem using a single translated multiplicative neuron. Neural Processing Letters 18(3), 233–238 (2003)

    Article  Google Scholar 

  17. Jankowski, N., Grochowski, M.: Comparison of instance selection algorithms. i. algorithms survey. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 598–603. Springer, Heidelberg (2004)

    Google Scholar 

  18. Jones, C., Sibson, R.: What is projection pursuit. Journal of the Royal Statistical Society A 150, 1–36 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  19. Kohonen, T.: Self-organizing maps. Springer, Heidelberg (1995)

    Google Scholar 

  20. Kordos, M., Duch, W.: Variable Step Search MLP Training Method. International Journal of Information Technology and Intelligent Computing 1, 45–56 (2006)

    Google Scholar 

  21. Liu, D., Hohil, M., Smith, S.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48, 477–488 (2002)

    Article  MATH  Google Scholar 

  22. Maass, W., Markram, H.: Theory of the computational function of microcircuit dynamics. In: Grillner, S., Graybiel, A.M. (eds.) Microcircuits,The Interface between Neurons and Global Brain Function, pp. 371–392. MIT Press, Cambridge (2006)

    Google Scholar 

  23. Marchand, M., Golea, M.: On learning simple neural concepts: from halfspace intersections to neural decision lists. Network: Computation in Neural Systems 4, 67–85 (1993)

    Article  Google Scholar 

  24. Mascioli, F.M.F., Martinelli, G.: A constructive algorithm for binary neural networks: The oil-spot algorithm. IEEE Transactions on Neural Networks 6(3), 794–797 (1995)

    Article  Google Scholar 

  25. Muselli, M.: On sequential construction of binary neural networks. IEEE Transactions on Neural Networks 6(3), 678–690 (1995)

    Article  MathSciNet  Google Scholar 

  26. Muselli, M.: Sequential constructive techniques. In: Leondes, C. (ed.) Optimization Techniques,of Neural Network Systems, Techniques and Applications, vol.2, pp. 81–144. Academic Press, San Diego (1998)

    Google Scholar 

  27. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, Cambridge (2007)

    MATH  Google Scholar 

  28. Sahami, M.: Learning non-linearly separable boolean functions with linear threshold unit trees and madaline-style networks. In: National Conference on Artificial Intelligence, pp. 335–341 (1993)

    Google Scholar 

  29. Shakhnarovish, G., Darrell, T., Indyk, P. (eds.): Nearest-Neighbor Methods in Learning and Vision. MIT Press, Cambridge (2005)

    Google Scholar 

  30. Stork, D.G., Allen, J.: How to solve the n-bit parity problem with two hidden units. Neural Networks 5, 923–926 (1992)

    Article  Google Scholar 

  31. Wilamowski, B., Hunter, D.: Solving parity-n problems with feedforward neural network. In: Proc. of the Int. Joint Conf. on Neural Networks (IJCNN 2003), vol. I, pp. 2546–2551. IEEE Computer Society Press, Los Alamitos (2003)

    Chapter  Google Scholar 

  32. Young, S., Downs, T.: Improvements and extensions to the constructive algorithm carve. In: Vorbrüggen, J.C., von Seelen, W., Sendhoff, B. (eds.) ICANN 1996. LNCS, vol. 1112, pp. 513–518. Springer, Heidelberg (1996)

    Google Scholar 

  33. Zollner, R., Schmitz, H.J., Wünsch, F., Krey, U.: Fast generating algorithm for a general three-layer perceptron. Neural Networks 5(5), 771–777 (1992)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, Nicolaus Copernicus University, Toruń, Poland

    Marek Grochowski & Włodzisław Duch

Authors
  1. Marek Grochowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Włodzisław Duch
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. of Computer Science, University of Malaga , Campus de Teatinos S/N, 29071, Málaga, Spain

    Leonardo Franco  & José M. Jerez  & 

  2. Centre for Computational Intelligence, School of Computing, De Montfort University, The Gateway , LE1 9BH, Leicester, UK

    David A. Elizondo

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Grochowski, M., Duch, W. (2009). Constructive Neural Network Algorithms That Solve Highly Non-separable Problems. In: Franco, L., Elizondo, D.A., Jerez, J.M. (eds) Constructive Neural Networks. Studies in Computational Intelligence, vol 258. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04512-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04512-7_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04511-0

  • Online ISBN: 978-3-642-04512-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation