Abstract
In this paper, the relationship between information and reasoning is investigated and a parallel reasoning method is proposed based on information theory, in particular the principle of minimum cross entropy. Some technical issues, such as multiple uncertain evidence, complicated constraints, small directed cycles and decomposition of underlying networks, are discussed. Some simple examples are also given to compare the method proposed here with other methods.
Similar content being viewed by others
References
P. Cheeseman, A method of computing generalized Bayesian probability values for expert systems, in:Proc. IJCAI 83, Karlsruhe, FRG (1983) pp. 198–202.
G.F. Cooper, NESTOR: a computer-based medical diagnostic aid that integrates causal and probabilistic knowledge, Report HPP-84-48, Stanford University (1984).
A.P. Dempster and A. Kong, Uncertain evidence and artificial analysis, J. Statist. Planning and Inference 20 (1988) 355–368.
P. Diaconis and S.L. Zabell, Updating subjective probability, J. Am. Statist. Ass. 77 (1982) 822–830.
S.A. Goldman and R.L. Rivest, A non-iterative maximum entropy algorithm, in:Uncertainty in Artificial Intelligence, vol. 2, eds. J.F. Lemmer and L.N. Kanal (North-Holland, 1988).
S.W. Hawking,A Brief History of Time: From the Big Bang to Black Holes (Bantam Press, 1988).
R. Jeffrey, Contributions to the theory of inductive probability, Ph.D. Thesis, Princeton, NJ, Princeton University, Department of Philosophy (1957).
H. Kiiveri, T.P. Speed and J.B. Carlin, Recursive causal models, J. Austral. Math. Soc. A36 (1984) 30–52.
A. Kong, Construction of a tree of cliques from a triangulated graph, Technical Report S-118, Department of Statistics, Harvard University (1986).
S. Kullback,Information Theory and Statistics (Dover, 1959, 1968) p. 37.
S.L. Lauritzen and D.J. Spiegelhalter, Local computations with probabilities on graphical structures and their application to experts systems, J.R. Statist. Soc. B50 (1988).
J.F. Lemmer, Generalized Bayesian updating of incompletely specified distributions, Large Scale Systems 5 (1983).
J.F. Lemmer and S.W. Barth, Efficient minimum information updating for Bayesian inferencing, in:Proc. Nation. Conf. on Artificial Intelligence, AAAI, Pittsburgh (1982).
J. Pearl, Fusion, propagation, and structuring in belief networks, Artificial Intelligence 29 (1986) 241–228.
J. Pearl, Jeffrey's rule, passage of experience and neo-Bayesianism, Tech. Rep. Computer Science Dpt.-860099 R-62-II, Computer Science Department (December 1988).
J. Pearl, Evidential reasoning using stochastic simulation, Artificial Intelligence 32 (1987) 245–257.
C.E. Shannon,The Mathematical Theory of Communication (University of Illinois Press, 1948).
J.E. Shore, Relative entropy, probabilistic inference, and AI, in:Uncertainty in Artificial Intelligence, eds. L.N. Kanal and J.F. Lemmer (North-Holland, 1986).
J. Shore and R.W. Johnson, Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross entropy, IEEE Trans. Infor. Theory IT-26 (1980) 26–37.
D.J. Spiegelhalter, Probabilistic reasoning in predictive expert systems, in:Uncertainty in Artificial Intelligence, eds. L.N. Kanal and J.F. Lemmer (North-Holland, 1986).
R.E. Tarjan and M. Yannakakis, Simple linear-time algorithms to test chordality of graphs, acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs, SIAM J. Compt. 13 (1984) 566–579.
W.X. Wen, MCE reasoning in recursive causal networks, Technical Report TR88/16, in:Proc. AAAI-88 Workshop on Uncertainty in AI (1988).
W.C. Wen, MCE properties of recursive causal networks and parallel reasoning, Technical Report, Computer Science, The University of Melbourne (1989) in preparation.
W.X. Wen, Parallel MCE reasoning and Boltzmann-Jeffrey machine networks, in:Proc. 3rd IEEE Int. Symp. on Intelligent Control, Arlington, VA, USA (Aug. 1988).
W.X. Wen, Probabilistic reasoning with multiple uncertain evidence, Technical Report 88/34, Computer Science, The University of Melbourne (July 1988).
W.X. Wen, Analytical and numerical methods for minimum cross entropy problems, Technical Report 88/26, Computer Science, The University of Melbourne (1988).
W.X. Wen, Directed cycles in causal networks, Technical Report 89/9, Computer Science, The University of Melbourne (Nov. 1988).
W.X. Wen, Parallel MCE reasoning in recursive causal networks, Technical Report TR88/15, Computer Science, The University of Melbourne (April 1988); an extended abstract was presented atAAAI-88 Workshop on Parallel Algorithms in AI.
N. Wermuth and S.L. Lauritzen, Graphic and recursive models for contingency tables, Biometrika 70 (1983) 537–552.
N. Wermuth, Linear recursive equations, covariance selection, and path analysis, J. Am. Statist. Assoc. 75 (1980) 963–972.
N. Wiener,Cybernetics (Wiley, 1948).
B.P. Wise, An experimental comparison of uncertain inference systems, Ph.D. Thesis, Engineering and Public Policy, Carnegie-Mellon University (June 1986).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wen, W.X. Information and probabilistic reasoning. Ann Math Artif Intell 2, 367–381 (1990). https://doi.org/10.1007/BF01531017
Issue Date:
DOI: https://doi.org/10.1007/BF01531017