A conditional-logical approach to minimum cross-entropy

Abstract

The principle of minimum cross-entropy (ME-principle) is often used in the AI-areas of knowledge representation and uncertain reasoning as an elegant and powerful tool to build up complete probability distributions when only partial knowledge is available. The inputs it may be applied to are a prior distribution P and some new information R, and it yields as a result the one distribution P ^* that satisfies R and is closest to P in an information-theoretic sense. More generally, it provides a ”best” solution to the problem ”How to adjust P to R?”

In this paper, we show in a rather direct and constructive manner that adjusting P to R by means of this principle follows a simple and intelligible conditional-logical pattern. The scheme that underlies ME-adjustment is made obvious, and in a generalized form, it provides a straightforward conditional-logical approach to the adaptation problem. We introduce the idea of a functional concept and show how the demands for logical consistency and representation invariance influence the functions involved. Finally, the ME-distribution arises as the only solution which follows the simple adaptation scheme given and satisfies these three assumptions. So a characterization of the ME-principle within a conditional-logical framework will have been achieved, and its logical mechanisms will be revealed clearly.