Skip to main content
SpringerLink
Log in

Challenges in Neural Computation

  • Fachbeitrag
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

This contribution contains a short history of neural computation and an overview about the major learning paradigms and neural architectures used today.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for Boltzmann machines. Cogn Sci 9(1):147–169

    Article  Google Scholar 

  2. Anlauf JK, Biehl M (1989) The adatron: an adaptive perceptron algorithm. Europhys Lett 10(7):687

    Article  Google Scholar 

  3. Anthony M, Bartlett PL (2002) Neural network learning—theoretical foundations. Cambridge University Press, Cambridge

    Google Scholar 

  4. Arlt W, Biehl M, Taylor A, Hahner S, Libé R, Hughes B, Schneider P, Smith D, Stiekema H, Krone N, Porfiri E, Opocher G, Bertherat J, Mantero F, Allolio B, Terzolo M, Nightingale P, Shackleton C, Bertagna X, Fassnacht M, Stewart P (2011) Urine steroid metabolomics as a biomarker tool for detecting malignancy in adrenal tumors. J Clin Endocrinol Metab 96(5):3775–3784. 2011-12-01 00:00:00.0

    Article  Google Scholar 

  5. Barto A, Sutton R, Anderson C (1983) Neuron-like adaptive elements that can solve difficult learning control problems. IEEE Trans Syst Man Cybern 13:834–846

    Article  Google Scholar 

  6. Bengio Y, Frasconi P (1993) Credit assignment through time: alternatives to backpropagation. In: Cowan JD, Tesauro G, Alspector J (eds) NIPS. Kaufmann, Los Altos, pp 75–82

    Google Scholar 

  7. Bengio Y, Lecun Y (2007) Large scale kernel machines. In: Bottou L, Chapelle O, Decoste D, Weston J (eds) Large-scale kernel machines, Chapter: Scaling learning algorithms towards AI. MIT Press, Cambridge

    Google Scholar 

  8. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166

    Article  Google Scholar 

  9. Bishop CM (2007) Pattern recognition and machine learning. Information science and statistics, 1st edn. Springer, Berlin. corr. 2nd printing edn, 2006

    Google Scholar 

  10. Bishop CM, Williams CKI (1998) Gtm: The generative topographic mapping. Neural Computat 10:215–234

    Article  Google Scholar 

  11. Bojer T, Hammer B, Koers C (2003) Monitoring technical systems with prototype based clustering. In: Verleysen M (ed) ESANN 2003

    Google Scholar 

  12. Bubeck S, von Luxburg U (2009) Nearest neighbor clustering: A baseline method for consistent clustering with arbitrary objective functions. J Mach Learn Res 10:657–698

    MathSciNet  MATH  Google Scholar 

  13. Bunte K, Biehl M, Hammer B (2012) A general framework for dimensionality reducing data visualization mapping. Neural Comput 24(3):771–804

    Article  MATH  Google Scholar 

  14. Carpenter G, Grossberg S (1987) Adaptive resonance theory

  15. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  16. Cottrell M, Fort J-C, Pagès G (1994) Two or three things that we know about the Kohonen algorithm. In: ESANN

    Google Scholar 

  17. Crespi A, Lachat D, Pasquier A, Ijspeert AJ (2008) Controlling swimming and crawling in a fish robot using a central pattern generator. Auton Robots 25(1–2):3–13

    Article  Google Scholar 

  18. Doya K (1995) Supervised learning in recurrent networks. In: Arbib M (ed) Handbook of brain theory and neural networks. MIT Press, Cambridge

    Google Scholar 

  19. Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211

    Article  Google Scholar 

  20. Frasconi P, Gori M, Sperduti A (1998) A general framework for adaptive processing of data structures. IEEE Trans Neural Netw 9(5):768–786

    Article  Google Scholar 

  21. Frénay B, Verleysen M (2011) Parameter-insensitive kernel in extreme learning for non-linear support vector regression. Neurocomputing 74(16):2526–2531

    Article  Google Scholar 

  22. Fukushima K (1988) Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Netw 1(2):119–130

    Article  Google Scholar 

  23. Gärtner T, Lloyd JW, Flach PA (2004) Kernels and distances for structured data. Mach Learn 57(3):205–232

    Article  MATH  Google Scholar 

  24. Giles CL, Omlin CW, Thornber KK (1999) Equivalence in knowledge representation: automata, recurrent neural networks, and dynamical fuzzy systems. Proc IEEE 87(9):1623–1640

    Article  Google Scholar 

  25. Hammer B (2001) Generalization ability of folding networks. IEEE Trans Knowl Data Eng 13(2):196–206

    Article  Google Scholar 

  26. Hammer B, Villmann T (2003) Mathematical aspects of neural networks. In: ESANN, pp 59–72

    Google Scholar 

  27. Hebb D (2002) The organization of behavior: a neuropsychological theory. Erlbaum, Hillsdale

    Google Scholar 

  28. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  29. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554–2558

    Article  MathSciNet  Google Scholar 

  30. Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  31. Hyvärinen A, Karhunen J, Oja E (2001) Independent component analysis. Adaptive and learning systems for signal processing, communications, and control. Wiley, New York

    Book  Google Scholar 

  32. ichi Amari S, Maginu K (1988) Statistical neurodynamics of associative memory. Neural Netw 1(1):63–73

    Article  Google Scholar 

  33. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69

    Article  MathSciNet  MATH  Google Scholar 

  34. Kohonen T, Schroeder MR, Huang TS (eds) (2001) Self-organizing maps, 3rd edn. Springer, New York

    MATH  Google Scholar 

  35. Lange S, Riedmiller M (2010) Deep learning of visual control policies. In: ESANN

    Google Scholar 

  36. Lawrence R (1997) Using neural networks to forecast stock market prices. Methods, 1–21

  37. Lee JA, Verleysen M (2010) Scale-independent quality criteria for dimensionality reduction. Pattern Recogn Lett 31(14):2248–2257

    Article  Google Scholar 

  38. Maass W, Orponen P (1998) On the effect of analog noise in discrete-time analog computations. Neural Comput 10(5):1071–1095

    Article  Google Scholar 

  39. Maass W, Sontag ED (1999) Analog neural nets with Gaussian or other common noise distribution cannot recognize arbitrary regular languages. Neural Comput 11(3):771–782

    Article  Google Scholar 

  40. Martinetz TM, Berkovich SG, Schulten KJ (1993) ‘Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Trans Neural Netw 4(4):558–569

    Article  Google Scholar 

  41. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biol 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  42. Micheli A (2009) Neural network for graphs: a contextual constructive approach. IEEE Trans Neural Netw 20(3):498–511

    Article  MathSciNet  Google Scholar 

  43. Micheli A, Sona D, Sperduti A (2000) Bi-causal recurrent cascade correlation. In: IJCNN (3), pp 3–8

    Google Scholar 

  44. Minsky M, Papert S (1969) Perceptrons: an introduction to computational geometry. MIT Press, Cambridge

    MATH  Google Scholar 

  45. Möller R, Hoffmann H (2004) An extension of neural gas to local PCA. Neurocomputing 62:305–326

    Article  Google Scholar 

  46. Oja E (1982) Simplified neuron model as a principal component analyzer. J Math Biol 15:267–273. doi:10.1007/BF00275687

    Article  MathSciNet  MATH  Google Scholar 

  47. Palm G, Schwenker F, Sommer FT, Strey A (1993) Neural associative memories. Biol Cybern 36:19–36

    Article  Google Scholar 

  48. Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1481–1497

    Article  Google Scholar 

  49. Ritter H, Martinetz T, Schulten K (1992) Neural computation and self-organizing maps—an introduction. Computation and neural systems series. Addison-Wesley, Reading

    MATH  Google Scholar 

  50. Rost B, Sander C (1993) Prediction of protein secondary structure at better than 70 accuracy. J Mol Biol 232:584–599

    Article  Google Scholar 

  51. Rückert U, Merényi E (2012) Parallel neural hardware: the time is right. In: Verleysen M (ed) ESANN’12

    Google Scholar 

  52. Rumelhart DE, Hinton GE, Williams RJ (1988) Learning representations by back-propagating errors. In neurocomputing: foundations of research, Chapter: Learning representations by back-propagating errors. MIT Press, Cambridge, pp 696–699

    Google Scholar 

  53. Sanger TD (1989) Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Netw 2:459–473

    Article  Google Scholar 

  54. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80

    Article  Google Scholar 

  55. Sejnowski TJ, Rosenberg CR (1986) NETtalk: a parallel network that learns to read aloud. Technical report JHU/EECS-86/01, Electrical Engineering and Computer Science, Johns Hopkins University, Baltimore, MD

  56. Siegelmann HT (2003) Neural and super-turing computing. Minds Mach 13(1):103–114

    Article  MATH  Google Scholar 

  57. Steil JJ (2007) Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning. Neural Netw 20(3):353–364

    Article  MATH  Google Scholar 

  58. Surhone L, Tennoe M, Henssonow S (2010) Dectalk. VDM Verlag Dr. Mueller AG & Co Kg

  59. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Adaptive computation and machine learning. MIT Press, Cambridge

    Google Scholar 

  60. Tiño P, Dorffner G (2001) Predicting the future of discrete sequences from fractal representations of the past. Mach Learn 45(2):187–217

    Article  MATH  Google Scholar 

  61. Tsang IW-H, Kwok JT-Y, Zurada JM (2006) Generalized core vector machines. IEEE Trans Neural Netw 17(5):1126–1140

    Article  Google Scholar 

  62. Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134–1142

    Article  MATH  Google Scholar 

  63. Vellido A, Martin-Guerrero J, Lisboa P (2012) Making machine learning models interpretable. In: Verleysen M (ed) ESANN’12

    Google Scholar 

  64. Venna J, Peltonen J, Nybo K, Aidos H, Kaski S (2010) Information retrieval perspective to nonlinear dimensionality reduction for data visualization. J Mach Learn Res 11:451–490

    MathSciNet  MATH  Google Scholar 

  65. Villmann T, Claussen JC (2006) Magnification control in self-organizing maps and neural gas. Neural Comput 18(2):446–469

    Article  MathSciNet  MATH  Google Scholar 

  66. Villmann T, Der R, Herrmann JM, Martinetz T (1994) Topology preservation in self-organizing feature maps: general definition and efficient measurement. In: Reusch B (ed) Fuzzy days. Informatik aktuell. Springer, Berlin, pp 159–166

    Google Scholar 

  67. von der Malsburg C (1973) Self-organization of orientation sensitive cells in the striate cortex. Kybernetik 14:85–100

    Article  Google Scholar 

  68. Werbos PJ (1994) The roots of backpropagation: from ordered derivatives to neural networks and political forecasting. Adaptive and learning systems for signal processing, communications and control series. Wiley-Interscience, New York

    Google Scholar 

  69. Widrow B, Hoff ME (1960) Adaptive switching circuits. In: 1960 IRE WESCON convention record, part 4. IRE, New York, pp 96–104

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CITEC Centre of Excellence, Bielefeld University, 33594, Bielefeld, Germany

    Barbara Hammer

Authors
  1. Barbara Hammer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Barbara Hammer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hammer, B. Challenges in Neural Computation. Künstl Intell 26, 333–340 (2012). https://doi.org/10.1007/s13218-012-0209-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-012-0209-0

Keywords

Search

Navigation