Abstract
An often used methodology for reasoning with probabilistic conditional knowledge bases is provided by the principle of maximum entropy (so-called MaxEnt principle) that realises an idea of least amount of assumed information and thus of being as unbiased as possible. In this paper we exploit the fact that MaxEnt distributions can be computed by solving nonlinear equation systems that reflect the conditional logical structure of these distributions. We apply the theory of Gröbner bases that is well known from computational algebra to the polynomial system which is associated with a MaxEnt distribution, in order to obtain results for reasoning with maximum entropy. We develop a three-phase compilation scheme extracting from a knowledge base consisting of probabilistic conditionals the information which is crucial for MaxEnt reasoning and transforming it to a Gröbner basis. Based on this transformation, a necessary condition for knowledge bases to be consistent is derived. Furthermore, approaches to answering MaxEnt queries are presented by demonstrating how inferring the MaxEnt probability of a single conditional from a given knowledge base is possible. Finally, we discuss computational methods to establish general MaxEnt inference rules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bochmann, A.: A Logical Theory of Nonmonotonic Inference and Belief Change. Springer (2001)
Buchberger, B.: An algorithm for finding the basis elements in the residue class ring modulo a zero dimensional polynomial ideal. J. Symb. Comput. 41(3–4), 475–511 (2006)
Cox, D.A., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra. Springer (2007)
Dukkipati, A.: Embedding maximum entropy models in algebraic varieties by Gröbner bases methods. In: Proceedings of the International Symposium on Information Theory, ISIT 2009. IEEE, pp. 1904–1908 (2009)
Dukkipati, A.: On maximum entropy and minimum KL-divergence optimization by Gröbner basis methods. Appl. Math. Comput. 218(23), 11,674–11,687 (2012)
Eichhorn, C., Diekmann, K., Kern-Isberner, G.: Properties of (nonmonotonic) inference relations with reflection on System Z and c-representations. Manuscript (2014)
Gärdenfors, P.: Knowledge in Flux: Modeling the Dynamics of Epistemic States. MIT Press, Cambridge (1988)
Hawthorne, J., Makinson, D.: The quantitative/quantitative watershed for rules of uncertain inference. Stud. Logica. 86(2), 247–297 (2007)
Jaynes, E.T.: Papers on Probability, Statistics and Statistical Physics. D. Reidel, Dordrecht (1983)
Kern-Isberner, G.: A note on conditional logics and entropy. Int. J. Approx. Reason. 19, 231–246 (1998)
Kern-Isberner, G.: Conditionals in nonmonotonic reasoning and belief revision. Lecture Notes in Artificial Intelligence LNAI 2087. Springer (2001)
Kern-Isberner, G., Thimm, M., Doherty, P.: A ranking semantics for first-order conditionals. In: De Raedt, L., Bessiere, C., Dubois, D., Frasconi, P., Heintz, F., Lucas, P. (eds.) Proceedings 20th European Conference on Artificial Intelligence no. 242 in Frontiers in Artificial Intelligence and Applications, ECAI-2012, pp. 456–461. IOS Press (2012)
Kern-Isberner, G., Wilhelm, M., Beierle, C.: Probabilistic knowledge representation using Gröbner basis theory. In: International Symposium on Artificial Intelligence and Mathematics (ISAIM 2014). January 6-8, 2014, Fort Lauderdale (2014)
Kraus, S., Lehmann, D.J., Magidor, M.: Nonmonotonic reasoning, preferential models and cumulative logics. Artif. Intell. 44(1-2), 167–207 (1990)
Lukasiewicz, T.: Weak nonmonotonic probabilistic logics. Artif. Intell. 168(1-2), 119–161 (2005)
Makinson, D.: Handbook of logic in artificial intelligence and logic programming General Patterns In Nonmonotonic Reasoning, vol. 3, pp. 35–110. Oxford University Press (1994)
Mayr, E.W.: Some complexity results for polynomial ideals. J. Complex. (1997)
Paris, J.B.: The Uncertain Reasoner’S Companion – A Mathematical Perspective. Cambridge University Press (1994)
Paris, J.B.: Common sense and maximum entropy. Synthese 117, 75–93 (1999)
Paris, J.B., Vencovská, A.: A note on the inevitability of maximum entropy. Int. J. Approx. Reason. 14, 183–223 (1990)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo (1988)
Rödder, W., Meyer, C.H.: Coherent knowledge processing at maximum entropy by SPIRIT. In: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence, UAI 1996, pp. 470–476 (1996)
Rödder, W., Reucher, E., Kulmann, F.: Features of the expert-system-shell SPIRIT. Log. J. IGPL 14(3), 483–500 (2006)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kern-Isberner, G., Wilhelm, M. & Beierle, C. Probabilistic knowledge representation using the principle of maximum entropy and Gröbner basis theory. Ann Math Artif Intell 79, 163–179 (2017). https://doi.org/10.1007/s10472-015-9457-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10472-015-9457-7