Abstract
Adaptive generalisation is the ability to use prior knowledge in the performance of novel tasks. Thus, if we are to model intelligent behaviour with neural nets, they must be able to generalise across task domains. Our objective is to elucidate the aetiology of transfer of information between connectionist nets. First, a method is described that provides a standardised score for the quantification of how much task structure a net has extracted, and to what degree knowledge has been transferred between tasks. This method is then applied in three simulation studies to examine Input-to-Hidden (IH) and Hidden-to-Output (HO) decision hyperplanes as determinants of transfer effects. In the first study, positive transfer is demonstrated between functions that require the vertices of their spaces to be divided similarly, and negative transfer between functions that require decision regions of different shapes. In the other two studies, input and output similarity are varied independently in a series of paired associate learning tasks. Further explanation of transfer effects is provided through the use of a new technique that permits better visualisation of the entire computational space by showing both the relative position of inputs in Hidden Unit space, and the HO decision regions implemented by the set of weights.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bartlett, F. C. (1932).Remembering. Cambridge: Cambridge University Press.
Bates, E. & Elman, J. L. (in press). Connectionism and the Study of Change. In M. H. Johnson (ed.),Brain Development and Cognition: A Reader. Oxford: Blackwells.
Bugelski, B. R. (1942). Interferences with Recall of Original Responses after Learning New Responses to Old Stimuli.Journal of Experimental Psychology 30: 368–379.
Ebbinghaus, H. (1885).Memory: A Contribution to Experimental Psychology (H. A. Ruger & C. E. Bussenius, Tr.) New York: Teachers College, Columbia University, 1913.
Gibson, E. J. (1941). Retroactive Inhibition as a Function of Degree of Generalisation Between Tasks.Journal of Experimental Psychology 28(2): 93–115.
Hebb, D. O. (1949).The Organisation of Behavior. New York: Wiley.
Hetherington, P. A. & Seidenberg, M. S. (1989). Is there ‘Catastrophic Interference’ in Connectionist Nets? In Proceedings 11th Annual Conference of Cognitive Science Society: Michigan.
James, W. (1890).Principles of Psychology. New York: Holt.
McCloskey, M. & Cohen, N. J. (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem.The Psychology of Learning and Motivation 24: 109–165.
Mundy, D. & Sharkey, N. E. (1992). Type Generalisations on Distributed Representations. In R. Trapl (ed.),Cybernetics and Systems Research, 1327–1334. Kluwer Academic Publishers: Dordrecht, The Netherlands.
Pavlov, I. P. (1927).Conditioned Reflexes (Translated by G. V. Anrep) London: Oxford.
Piaget, J. (1952).The Origins of Intelligence. Translated by M. Cook. New York: International Universities.
Pratt, L. Y. (1992). Non-Literal Transfer of Information among Inductive Learners. Computer Science Department, Rutgers University Working Paper, May 1992.
Pratt, L. Y. & Kamm, C. A. (1991). Improving a Phoneme Classification Task Through Problem Decomposition. In Proceedings of the International Joint Conference on Neural Networks (IJCNN-91).
Osgood, C. E. (1949). The Similarity Paradox in Human Learning: A Resolution.Psychological Review 56: 132–143.
Sharkey, N. E. & Partridge, D. (1992). The Statistical Independence of Network Generalisation: An Application in Software Engineering. In P. G. Lisboa & M. J. Taylor (eds.),Neural Networks: Techniques and Applications. Ellis Horwood: Chichester, UK.
Sharkey, N. E. & Sharkey, A. J. C. (in press). Prestructured Neural Nets and the Transfer of Knowledge. In G. Orchard (ed.),Proceedings of Second Irish Neural Networks Conference. Adam Hilger.
Thorndyke, E. L. & Woodworth, R. S. (1901). The Influence of Improvement in One Mental Function upon the Efficiency of Other Functions. I, I, IIPsychological Review 8: 247–261, 384–395, 553–564.
Volkman, A. (1858). Úber den Einfluss der Übung auf das Erkennen raumlicher Distanzen. Berichte Sachisischer Gesellshaft der Wissenschaftern,Mathematisch-Physiche Klasse 10: 38–69.
White, H. (1992).Artificial Neural Networks: Approximation and Learning Theory. Blackwell, Oxford, UK.
Author information
Authors and Affiliations
Additional information
This research was supported by an award from the Economic and Social Research Council, Grant No R000233441. An earlier version of this paper appears in the Proceedings of the Second Irish Networks Conference, Belfast 1992.
Rights and permissions
About this article
Cite this article
Sharkey, N.E., Sharkey, A.J.C. Adaptive generalisation. Artif Intell Rev 7, 313–328 (1993). https://doi.org/10.1007/BF00849058
Issue Date:
DOI: https://doi.org/10.1007/BF00849058