Abstract
BiDirectional Neural Networks (BDNN) are based on Multi Layer Perceptrons trained by the error back-propagation algorithm. They can be used as both associative memories and to find the centres of clusters.
One of the major challenges in neural network research is data representation. We have used cluster centroids obtained by BDNNs and some heuristic techniques to achieve good representations. This is the key factor in reducing generalisation error. Evaluation of these methods is done by statistical learning theory supported by experimental results. A variety of data sets from real-world problems have been used to support the results of our methods.
The results are consistent with the Vapnik-Chervonenkis bounds. Our methods can be considered as efficient means of designing the required bias in solving dynamic and complex learning systems and to increase their expected performance.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
Amirikian, B., and Nishimura, H. 1994. What Size Network Is Good for Generalisation of a Specific Task of Interest? Neural Networks, Vol. 7, No. 2, 321–329.
Anderson, J.A. 1988. Cognitive and Psychological Computation with Neural Models. IEEE Transactions on Systems, Man, and Cybernetics 13, 799–815.
Baum, E.B., and Haussler, D. 1989. What Size Net Gives Valid Generalisation? Neural Computation 1, 151–160.
Blumer, A., Ehrenfeucht, A., Haussler D., and Warmuth M. 1989. Leamability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4), pp. 929–965.
Bustos, RA and Gedeon, TD Decrypting Neural Network Data: A GIS Case Study, Proceedings International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA), Alès, 1995.
Cohn, D. and Tesauro, G. 1992. How Tight are the Vapnik-Chervonenkis Bounds? Neural Computation 4, 249–269.
Ehrenfeucht, A., Haussler, D., Kearns, M., and Valiant, L. 1988. A General Lower Bound on the Number of Examples Needed for Learning. In Proceedings of the 1988 Workshop on Computational Learning Theory. San Mateo, CA, Morgan Kaufmann.
Gedeon, T.D. and Bowden, T.G. 1992. Heuristic pattern reduction. International Joint Conference on Neural Networks, Beijing, pp. 449–453.
Gedeon, TD and Good, RP Interactive modelling of a neural network model of GDP, Proceedings International Conference on Modelling and Simulation, pp. 355–360, Perth, 1993.
Gedeon, T.D. and Harris, D. Network Reduction Techniques, Proceedings International Conference on Neural Networks Methodologies and Applications, AMSE, vol. 1, pp. 119–126, San Diego, 1991.
Gedeon, T.D. and Turner, H. 1993. Explaining student grades predicted by a neural network. Proceedings International Joint Conference on Neural Networks, pp. 609–612, Nagoya.
Geman, S., Bienenstock, E., and Doursat R. 1992. Neural Networks and the Bias/Variance Dilemma. Neural Computation 4, 1–58.
Haussler, D., Littlestone, N., and Warmuth, M. 1990. Predicting 0,1-Functions on Randomly Drawn Points. Technical Report UCSC-CRL-90-54, University if California at Santa Cruz.
Hecht-Nielsen, R. 1987. Counterpropagation Networks. Applied Optics, vol. 26, no. 3, 4979–4984.
Kosko, B. 1988. Bidirectional Associative Memories. IEEE Transactions on Systems, Man, ami Cybernetics, vol. SMC-18, 49–60.
Kosko, B. 1992. Neural Networks and Fuzzy Systems.
Kruschke, J.K. 1988. Creating Local and Distributed Bottlenecks in Hidden Layers of Back-Propagation Networks. Processing of the 1988 Connectionist Summer School. Carnegie-Mellon University, Pittsburgh, PA: Morgan Kaufmann.
Maass, W. 1994. Neural Nets with Superlinear VC-Dimension. Neural Computation 6, 877–884.
Nejad, A.F., Gedeon T.D. 1994. BiDirectional MLP Neural Networks. International Symposium on Artificial Neural Networks. Tainan, Taiwan, pp. 308–313.
Nejad, A.F., Gedeon T.D. 1995. Analyser Neural Networks. International Workshop on Applications of Artificial Neural Networks to Telecommunications. Stockholm, Sweden.
Rumelhart, D.E., Hinton, G.E., Williams, R.J. 1986. Learning internal representations by error propagation. in Rumelhart, D.E., McClelland, Parallel Distributed Processing, Vol. 1, MIT Press.
Slade, P. and Gedeon, T.D. 1993. Bimodal Distribution Removal, Proceedings IWANN International Conference on Neural Networks, Barcelona. also in Mira, J., Cabestany, J. and Prieto, A. 1993. New Trends in Neural Computation, pp. 249–254, Springer Verlag, Lecture Notes in Computer Science, vol. 686.
Valiant, L.G. 1984. A Theory of the Learnable. Communications of the ACM, 27, 1134–1142
Vapnik, V.N. 1982. Estimation of Dependencies Based on Empirical Data. Spring-Verlag, New York.
Weigend, A., Rumelhart, D., and Huberman, B. 1991. Generalisation by Weight Elimination with Application to Forecasting. In Advances in Neural Information Processing Systems 3, R. Lippmann, J. Moody, and D. Touretzky, eds. Morgan Kaufmann, Denver, Co.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1995 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nejad, A.F., Gedeon, T.D. (1995). Bidirectional Neural Networks reduce generalisation error. In: Mira, J., Sandoval, F. (eds) From Natural to Artificial Neural Computation. IWANN 1995. Lecture Notes in Computer Science, vol 930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59497-3_221
Download citation
DOI: https://doi.org/10.1007/3-540-59497-3_221
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-59497-0
Online ISBN: 978-3-540-49288-7
eBook Packages: Springer Book Archive