Abstract
This paper deals with an important problem with large and complex Bayesian networks. Exact inference in these networks is simply not feasible owing to the huge storage requirements of exact methods. Markov chain Monte Carlo methods, however, are able to deal with these large networks but to do this they require an initial legal configuration to set off the sampler. So far nondeterministic methods such as forward sampling have often been used for this, even though the forward sampler may take an eternity to come up with a legal configuration. In this paper a novel algorithm will be presented that allows a legal configuration in a general Bayesian network to be found in polynomial time in almost all cases. The algorithm will not be proved deterministic but empirical results will demonstrate that this holds in most cases. Also, the algorithm will be justified by its simplicity and ease of implementation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Gelfand, A. E. and Smith, A. F. M. (1990) Sampling-based approaches to calculating marginal densities. Journal of American Statistical Association, 85(410), 398–409.
Gelman, A. and Rubin, D. B. (1992) Inference from iterative simulation using single and multiple sequences (with discussion). Statistical Science, 7, 457–511.
Geman, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–41.
Geyer, C. J. (1991) Markov chain Monte Carlo maximum likelihood, in Computing Science and Statistics, Proceedings of the 23rd Symposium on the Interface, pp. 156–63.
Geyer, C. J. (1992) Practical Markov Chain Monte Carlo. Statistical Science, 7(4), 473–511.
Geyer, C. J. and Thompson, E. A. (1995) Annealing Markov chain Monte Carlo with applications to ancestral inference. Journal of American Statistical Association, 90(431), 909–20.
Gilberg, A., Gilberg, L., Gilberg, R. et al. (1978) Polar Eskimo Genealogy, Vol. 203No. 4 of Meddelelser om Grønland, Nyt Nordisk Forlag, Arnold Busck, København.
Hastings, W. K. (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109.
Heath, S. C. (1998) Generating consistent genotypic configurations for multi-allelic loci and large complex pedigrees. Human Heredity, 48, 1–11.
Jensen, C. S. (1997) Blocking Gibbs Sampling for Inference in Large and Complex Bayesian Networks with Applications in Genetics, PhD Thesis, Department of Computer Science, Aalborg University, Denmark.
Jensen, C. S. and Kong, A. (1996) Linkage analysis in large pedigrees with many loops — blocking Gibbs, in European Mathematical Genetics Meeting, Clementi, M. and Forabosco, P. (eds), Cooperativa Libraria Editrice Università di Padova, pp. 84–98.
Jensen, C. S. and Sheehan, N. (1997) Problems with the determination of the noncommunicating classes for MCMC applications in pedigree analysis. Technical Report R–97–5004, Department of Computer Science, Aalborg University, Denmark. Accepted by Biometrics for publication in 1998.
Jensen, C. S., Kong, A. and Kjærulff, U. (1995) Blocking-Gibbs sampling in very large probabilistic expert systems. International Journal of Human-Computer Studies, 42, 647–66.
Jensen, F. V. (1988) Junction trees and decomposable hypergraphs. Research report, Judex Datasystemer A/S, Aalborg, Denmark.
Kong, A. (1991) Efficient methods for computing linkage likelihoods of recessive diseases in inbred pedigrees. Genetic Epidemiology, 8, 81–103.
Lauritzen, S. L. (1992) Propagation of probabilities, means and variances in mixed graphical association models. Journal of American Statistical Association, 87(420), 1098–1108.
Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50(2), 157–224.
Marinari, E. and Parisi, G. (1992) Simulated tempering: a new Monte Carlo scheme. Europhysics Letters, 19, 451–58.
Metropolis, N., Rosenbluth, A., Rosenbluth, M. et al. (1953) Equations of state calculations by fast computing machines. Journal of Chemistry and Physics, 21, 1087–91
Pearl, J. (1986) Fusion, propagation, and structuring in belief networks. Artificial Intelligence, 29, 241–88
Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Series in Representation and Reasoning, Morgan Kaufmann Publishers, San Mateo, California.
Sheehan, N. and Thomas, A. (1993) On the irreducibility of a Markov chain defined on a space of genotype configurations by a sampling scheme. Biometrics, 49, 163–75.
Shenoy, P. P. and Shafer, G. R. (1990) Axioms for probability and belief-function propagation, in Uncertainty in Artificial Intelligence 4, Shachter, R. D., Levitt, T. S., Kanal, L. N., and Lemmer, J. F. (eds), Elsevier Science Publishers B. V. (North-Holland), Amsterdam, pp. 169–98.
Smith, A. F. M. and Roberts, G. O. (1993) Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 55(1), 5–23.
Thomas, A. (1988) Drawing pedigrees. IMA Journal of Mathematics Applied in Medicine & Biology, 5, 201–13.
Thomas, A., Spiegelhalter, D. J. and Gilks, W. R. (1992) BUGS: A program to perform Bayesian inference using Gibbs sampling, in Bayesian Statistics 4, Bernardo, J. M., Berger, J. O., Dawid, A. P. and Smith, A. F. M. (eds), Clarendon Press, Oxford, UK, pp. 837–42.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Jensen, C.S. A simple method for finding a legal configuration in complex Bayesian networks. Statistics and Computing 8, 243–251 (1998). https://doi.org/10.1023/A:1008909428052
Issue Date:
DOI: https://doi.org/10.1023/A:1008909428052