Abstract
Theknowledge transfer problem in artificial intelligence consists of finding effective ways to elicit information from a human expert and represent it in a form suitable for use by an expert system. One approach to formalizing and guiding this knowledge transfer process for certain types of expert systems is to use psychometric scaling methods to analyze data on how the human expert compares or groups solutions. For example, Butler and Corter [1] obtained judgments of thesubstitutability of solutions from an expert, then analyzed the resulting data via techniques for fitting trees and extended trees [2]. The expert's interpretation of certain aspects of the solutions were directly encoded as production rules, allowing rapid prototyping. In this paper we consider the problem of combining information from multiple experts. We propose the use of three-way or “individual differences” multidimensional scaling, tree-fitting, and unfolding models to analyze two types of data obtainable from the multiple experts: judgments of the substitutability of pairs of solutions, and judgments of the appropriateness of specific solutions to specific problems. An application is described in which substitutability data were obtained from three experts and analyzed using the SINDSCAL program [3] for three-way multidimensional scaling [4].
Similar content being viewed by others
References
K.A. Butler and J.E. Corter, Use of psychometric scaling methods for knowledge acquisition: A case study, in:Artificial Intelligence and Statistics, ed. W.A. Gale (Addison-Wesley, New York, 1986).
J.E. Corter and A. Tversky, Extended similarity trees, Psychometrika 51 (1986) 429.
S. Pruzansky, How to use SINDSCAL: A computer program for individual differences in multidimensional scaling, unpublished paper, Bell Telephone Laboratories, Murray Hill, NJ (1975).
J.D. Carroll and J.J. Chang, Analysis of individual differences in multidimensional scaling via ann-way generalization of Eckart-Young decomposition, Psychometrika 35 (1970) 283
A.L. Kidd,Knowledge Acquisition for Expert Systems: A Practical Handbook (Plenum, New York, 1987).
D.W. Rolston,Principles of Artificial Intelligence and Expert Systems Development (McGraw-Hill, New York, 1988).
W.J. Clancey, Classification problem solving, in:AAAI-84: Proc. National Conf. on Artificial Intelligence, Los Angeles (1972).
M.L.G. Shaw, PLANET: Some experience in creating an integrated system for repertory grid applications on a microcomputer, Int. J. Man-Machine Studies 17 (1982) 345.
B.R. Gaines and M.L.G. Shaw, New directions in the analysis and interactive elicitation of personal construct systems, in:Recent Advances in Personal Construct Theory, ed. M.L.G. Shaw (Academic, New York, 1981).
M.L.G. Shaw and B.R. Gaines, A computer aid to knowledge engineering, in:Proc. British Computer Society Conf. on Expert Systems, Cambridge (1983).
A. Tversky, Features of similarity, Psychol. Rev. 84 (1977) 327.
N.M. Cooke and J.E. McDonald, The application of psychological scaling techniques to knowledge elicitation for knowledge-based systems, Int. J. Man-Machine Studies 26 (1987) 533.
J.G. Gammack, Different techniques and different aspects on declarative knowledge, in:Knowledge Acquisition for Expert Systems: A Practical Handbook, ed. A.L. Kidd (Plenum, New York, 1987).
J.H. Boose, Rapid acquisition and combination of knowledge from multiple experts in the same domain, in:Proc. 2nd Conf. on Artificial Intelligence Applications (1985).
S. Mittal and C.L. Dym, Knowledge acquisition from multiple experts, The AI Magazine (1985) 32.
J.D. Carroll, Spatial, non-spatial, and hybrid models for scaling, Psychometrika 41 (1976) 439.
J.D. Carroll and S. Pruzansky, Discrete and hybrid scaling models, in:Similarity and Choice, eds. E.D. Lantermann and H. Feger (Huber, Bern, 1980).
J.B. Kruskal, F.W. Young and J.B. Seery, How to use KYST-2, a very flexible program to do multidimensional scaling and unfolding, unpublished paper, Bell Telephone Laboratories, Murray Hill, NJ (1977).
Y. Takane, F.W. Young and J. De Leeuw, Nonmetric individual differences multidimensional scaling: an alternating least-squares method with optimal scaling features, Psychometrika 42 (1976) 7.
S.C. Johnson, Hierarchical clustering schemes, Psychometrika 32 (1967) 241.
S. Sattath and A. Tversky, Additive similarity trees, Psychometrika 42 (1977) 319.
J.E. Corter, ADDTREE/P: A PASCAL program for fitting additive trees based on Sattath and Tversky's ADDTREE algorithm, Behavior Res. Meth. Instr. 14 (1982) 353.
G. De Soete, A least-squares algorithm for fitting additive trees to proximity data, Psychometrika 48 (1983) 621.
R.N. Shepard and P. Arabie, Additive clustering: Representation of similarities as combinations of discrete overlapping properties, Psychol. Rev. 86 (1979) 87.
P. Arabie and J.D. Carroll, MAPCLUS: A mathematical programming approach to fitting the ADCLUS model, Psychometrika 45 (1980) 211.
S. Sattah and A. Tversky, On the relation between common and distinctive feature models, Psychol. Rev. 94 (1987) 16.
C. Coombs,A Theory of Data (Wiley, 1964).
J.D. Carroll, Individual differences and multidimensional scaling, in:Multidimensional Scaling: Theory and Applications in the Behavioral Sciences vol. 1, eds. R.N. Shepard, A. K. Romney and S.B. Nerlove (Seminar Press, New York, 1972).
J.D. Carroll, Models and methods for multidimensional analysis of preferential choice (or other dominance) data, in:Similarity and Choice, eds. E.D. Lanterman and H. Feger (Huber, Bern, 1980).
G. De Soete, W.S. DeSarbo, G. Furnas and J.D. Carroll, The estimation of ultrametric and path length trees from rectangular proximity data, Psychometrika 49 (1984) 289.
J.D. Carroll, L.A. Clark and W.S. DeSarbo, The representation of three-way proximity data by single and multiple tree structure models, J. Classification 1 (1984) 25.
J.D. Carroll and P. Arabie, INDCLUS: An individual differences generalization of the ADCLUS model and the MAPCLUS algorithm, Psychometrika 48 (1983) 157.
W.S. DeSarbo and J.D. Carroll, Three-way metric unfolding via alternating weighted leastsquares, Psychometrika 50 (1985) 275.
G. De Soete and J.D. Carroll, Ultrametric tree representations of three-way three-mode data, in:Analysis of Multi-Way Data Matrices, eds. R. Coppi and S. Bolasco (North-Holland, Amsterdam, in press).
J.D. Carroll and M. Wish, Models and methods for three-way multidimensional scaling, in:Contemporary Developments in Mathematical Psychology, vol. 2, eds. D.H. Krantz, R.C. Atkinson, R.D. Luce and P. Suppes (W.H. Freeman, San Francisco, 1974).
S. Pruzansky, A. Tversky and J.D. Carroll, Spatial vs tree representations of proximity data, Psychometrika 47 (1982) 3.
F. Critchley and W. Heiser, Hierarchical trees can be perfectly scaled in one dimension, J. Classification 5 (1988) 5.
M. Chi, P.J. Feltovich and R. Glaser, Categorization and representation of physics problems by experts and novices, Cognitive Sci. 5 (1981) 121.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Corter, J.E., Carroll, J.D. Potential applications of three-way multidimensional scaling and related techniques to integrate knowledge from multiple experts. Ann Math Artif Intell 2, 77–92 (1990). https://doi.org/10.1007/BF01530998
Issue Date:
DOI: https://doi.org/10.1007/BF01530998