Loading [MathJax]/extensions/TeX/ieeemacros.js
Properties of cross-entropy minimization | IEEE Journals & Magazine | IEEE Xplore

Properties of cross-entropy minimization


Abstract:

The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) is a general method of inference about an unknown probability den...Show More

Abstract:

The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) is a general method of inference about an unknown probability density when there exists a prior estimate of the density and new information in the form of constraints on expected values. Various fundamental properties of cross-entropy minimization are proven and collected in one place. Cross-entropy's well-known properties as an information measure are extended and strengthened when one of the densities involved is the result of cross-entropy minimization. The interplay between properties of cross-entropy minimization as an inference procedure and properties of cross-entropy as an information measure is pointed out. Examples are included and general analytic and computational methods of finding minimum cross-entropy probability densities are discussed.
Published in: IEEE Transactions on Information Theory ( Volume: 27, Issue: 4, July 1981)
Page(s): 472 - 482
Date of Publication: 06 January 2003

ISSN Information:


References

References is not available for this document.