Abstract:
Suppose that, for purposes of inductive inference or choosing an optimal decision, we wish to select a single distribution P* to act as representative of a class /spl Gam...Show MoreMetadata
Abstract:
Suppose that, for purposes of inductive inference or choosing an optimal decision, we wish to select a single distribution P* to act as representative of a class /spl Gamma/ of such distributions. The maximum entropy principle ("MaxeEnt") (Jaynes 1989; Csiszar 1991) is widely applied for this purpose, but its rationale has often been controversial (Shimony 1985; Seidenfeld 1986). Here we emphasize and generalize a reinterpretation of the maximum entropy principle (Topsoe (1979); Walley (1991); Grunwald (1998)): that the distribution P* that maximizes the entropy over /spl Gamma/ also minimizes the worst-case expected logarithmic score (log loss). In the terminology of decision theory (Berger 1985), P* is a robust Bayes, or /spl Gamma/-minimax, act, when loss is measured by the log loss. This gives a decision-theoretic justification for maximum entropy.
Published in: Proceedings of the IEEE Information Theory Workshop
Date of Conference: 25-25 October 2002
Date Added to IEEE Xplore: 06 January 2003
Print ISBN:0-7803-7629-3