On the combinatorial structure of Bayesian learning with imperfect information

Abstract

Apart from the enumerative nature of event frequencies, the connection between Bayesian theory and combinatories does not seem to be an obvious one. In this paper, I study the underlying combinatorial structure of a particular class of Bayesian revision problems. This class is characterized by two properties. First, the initial Bayesian prior is taken to be Dirichlet. Second, the sequence of observations generated by the stochastic process in question, assumed i.i.d., is taken to convey imperfect information in general.

In the body of the paper, I demonstrate that the combinatorial features of interest surface during the application of Bayes' Rule. This gives rise to a sequence of posterior distributions having strong combinatorial attributes. Not unexpectedly, the first moments of the posteriors turn out to have strong combinatorial attributes as well; and these are of special interest.

Finally, the analysis serves to generalize the well-known conjugate family property for Dirichlet distributions (see DeGroot [1]) from the case when observations convey perfect information to the case where they convey imperfect information.