Abstract
The paper offers a critical analysis of the procedure observed in many applications of neural networks. Given a problem to be solved, a favorite NN-architecture is chosen and its parameters tuned with some standard training algorithm, but without taking in consideration relevant features of the problem or possibly its interdisciplinary nature. Three relevant benchmark problems are discussed to illustrate the thesis that “brute force solving is not the same as understanding”.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aizenberg, I.: Solving the parity n problem and other nonlinearly separable problems using a single universal binary neuron. Computational Intelligence. In: Reusch, B. (ed.) Theory and Applications, Springer, Berlin (2006)
Aizenberg, I., Moraga, C.: Multilayer feedforward neural network based on multi-valued neurons (MLMVN) and a backpropagation learning algorithm. Soft Computing 11(2), 169–183 (2007)
Alvarez-Sánchez, J.R.: Injecting knowledge into the solution of the two-spiral problem. Neural Computing and Applications 8, 265–272 (1999)
Allende, H., Moraga, C., Salas, R.: Artificial neural networks in forecasting: A comparative analysis. Kybernetika 38(6), 685–707 (2002)
Box, G.E., Jenkins, G.M.: Time series analysis: forecasting and control, Holden-Day, Oakland CA, USA (1976)
Chow, T.W.S., Leung, C.T.: Nonlinear autoregressive integrated neural network model for short-term load forecasting. IEE Proc. Gener. Transm. Distrib. 143(5), 500–506 (1996)
Durbin, R., Rumelhart, D.: Product units: A computationally powerful and biologically plausible extension to backpropagation networks. Neural Computation 1, 133–142 (1989)
Fahlman, S.E., Lebiere, C.: The Cascade Correlation Learning Architecture. In: Touretzky, S. (ed.) Advances in Neural Information Processing Systems, Morgan Kaufmann, San Francisco (1990)
Funahashi, K.I.: On the approximate realization of continuous mappings by neural networks. Neural Networks 2(3), 183–192 (1989)
Han, J., Moraga, C.: The Influence of the Sigmoid Function Parameters on the Speed of Backpropagation Learning. In: Sandoval, F., Mira, J. (eds.) From Natural to Artificial Neural Computation. LNCS, vol. 930, pp. 195–201. Springer, Heidelberg (1995)
Hartman, E.J., Keeler, J.D., Kowalski, J.M.: Layered neural networks with Gaussian hidden units as universal approximators. Neural Computation 2, 210–215 (1990)
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward neural networks are universal approximators. Neural Networks 2(5), 359–366 (1989)
Igel, Ch., Huesken, M.: Improving the Rprop Learning Algorithm. In: Proc. 2nd Int. Symposium on Neural Computation, pp. 115–121. Academic Press, London (2000)
Lang, K.J., Witbrock, M.J.: Learning to tell two spirals apart. In: Proceedings of the 1988 Connectionist Models Summer School, Morgan Kaufmann, San Francisco (1988)
Leerink, L.R., Giles, C.L., Horne, B.G., Marwan, A.J.: Learning with product units. Advances in Neural Information Processing. NIPS-94, 537–544 (1994)
Martin, R.D., Smarov, A., Vandaele, W.: Robust methods for ARIMA models. In: Zellner, A. (ed.) Proc. Conf. applied time series analysis of economic data, ASA-Census-NBER, pp. 153–169 (1983)
Mizutani, E., Dreyfus, S.E.: MLP’s hidden-node saturations and insensitivity to initial weights in two classification benchmark problems: parity and two spirals. In: Proc. IEEE Intl. Joint Conf. on Neural Networks, pp. 2831–2836. IEEE Computer Society Press, Los Alamitos (2002)
Minor, J.M.: Parity with two layer feedforward net. Neural Networks 6, 705–707 (1993)
Moraga, C., Han, J.: Problem Solving =/= Problem understanding. In: Proceedings XVI International Conference of the Chilean Computer Science Society, pp. 22–30. SCCC–Press, Santiago (1996)
Müller, K.-R., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An introduction to Kernel-based learning algorithms. In: Hu, Y.H., Hwang, Y.-N. (eds.) Chapter 4 of Handbook of Neural Networks Signal Processing, CRC-Press, Boca Raton, USA (2002)
Müller, K.-R., Smola, A., Rätsch, G., Schölkopf, B., Kohlmorgen, J., Vapnik, V.: Using Support Vector Machines for Time Series Prediction. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 999–1004. Springer, Heidelberg (1997)
Rumelhart, D.E., McClelland, J.L.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press, Cambridge (1986)
Salas, R.: Private communication, 2002 and 2007
Sontag, E.D.: Feedforward nets for interpolation and classification. Jr. Comput. Systems Science 45, 20–48 (1992)
Stork, D.G., Allen, J.D.: How to solve the N-bit parity problem with two hidden units. Neural Networks 5, 923–926 (1992)
Wieland, A.: Two spirals. CMU Repository of Neural Network Benchmarks (1988), http://www.bolz.cs.cmu.edu/benchmarks/two-spirals.html
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Moraga, C. (2007). Design of Neural Networks. In: Apolloni, B., Howlett, R.J., Jain, L. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2007. Lecture Notes in Computer Science(), vol 4692. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74819-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-74819-9_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74817-5
Online ISBN: 978-3-540-74819-9
eBook Packages: Computer ScienceComputer Science (R0)