Skip to main content
SpringerLink
Log in

Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

In this paper we consider the optimal decomposition of Bayesian networks. More concretely, we examine empirically the applicability of genetic algorithms to the problem of the triangulation of moral graphs. This problem constitutes the only difficult step in the evidence propagation algorithm of Lauritzen and Spiegelhalter (1988) and is known to be NP-hard (Wen, 1991). We carry out experiments with distinct crossover and mutation operators and with different population sizes, mutation rates and selection biasses. The results are analysed statistically. They turn out to improve the results obtained with most other known triangulation methods (Kjærulff, 1990) and are comparable to results obtained with simulated annealing (Kjærulff, 1990; Kjærulff, 1992).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Banzhaf, W. (1990) The ‘molecular’ traveling salesman. Biological Cybernetics, 64, 7–14.

    Google Scholar 

  • Chakraborty, U. K. and Dastidar, D. G. (1993) Using reliability analysis to estimate the number of generations to convergence in genetic algorithms. Information Processing Letters, 46, 199–209.

    Google Scholar 

  • Chavez, R. M. and Cooper, G. F. (1990) A randomized approximation algorithm for probabilistic inference on Bayesian belief networks. Networks, 20, 661–85.

    Google Scholar 

  • Cooper, G. F. (1990) The computational complexity of probabilistic inference using Bayesian belief networks. Artificial Intelligence, 42, 393–405.

    Google Scholar 

  • Dagum, P. and Luby, M. (1993) Approximating probabilistic inference in Bayesian belief networks is NP-hard. Artificial Intelligence, 60, 141–53.

    Google Scholar 

  • Dagum, P. and Horvitz, E. (1993) A Bayesian analysis of simulation algorithms for inference in belief networks. Networks, 23, 499–516.

    Google Scholar 

  • Davis, L. (1985) Applying adaptive algorithms to epistatic domains. In Proceedings International Joint Conference on Artificial Intelligence, Los Angeles, CA, pp. 162–4.

  • Davis, L. (1991) Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.

    Google Scholar 

  • De Jong, K. A. (1975) An analysis of the behaviour of a class of genetic adaptive systems. Ph.D. Dissertation, University of Michigan.

  • Eiben, A. E., Aarts, E. H. L. and Van Hee, K. M. (1990) Global convergence of genetics algorithms: an infinite Markov chain analysis. Computing Science Notes, Eindhoven University of Technology, The Netherlands.

    Google Scholar 

  • Fogel, D. B. (1990) A parallel processing approach to a multiple traveling salesman problem using evolutionary programming. In Proceedings of the Fourth Annual Parallel Processing Symposium, Fullerton, CA, pp. 318–26.

  • Fox B. R. and McMahon, M. B. (1991) Genetic operators for sequencing problems. In Foundations of Genetic Algorithms: First Workshop on the Foundations of Genetic Algorithms and Classifier Systems (G. Rawlins, ed.), pp. 284–300. Morgan Kaufmann, Los Altos, CA.

  • Fujisawa, T. and Orino, H. (1974) An efficient algorithm of finding a minimal triangulation of a graph. In IEEE International Symposium on Circuits and Systems, San Francisco, California, pp. 172–5.

  • Fung, R. M. and Chang, K. C. (1990) Weighing and integrating evidence for stochastic simulation in Bayesian networks. In Uncertainty in Artificial Intelligence 5, (M. Henrion, R. D. Shachter, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–20, Elsevier, Amsterdam.

    Google Scholar 

  • Goldberg, D. E. (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading, MA.

    Google Scholar 

  • Goldberg, D. E. and Lingle, Jr. R. (1985) Alleles, loci and the traveling salesman problem. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, Pittsburgh, PA, pp. 154–9.

  • Grefenstette, J., Gopal, R., Rosmaita, B. and Van Gucht, D. (1985) Genetic algorithms for the traveling salesman problem. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, Pittsburgh, PA, pp. 160–5.

  • Henrion, M. (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Uncertainty in Artificial Intelligence 2, (J. F. Lemmer and L. N. Kanal, eds), pp. 149–63, North-Holland, Amsterdam.

  • Holland, J. (1975) Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor.

    Google Scholar 

  • Homaifar, A. and Guan, S. (1991) A new approach on the traveling salesman problem by genetic algorithm. Technical Report, North Carolina A & T State University.

    Google Scholar 

  • Hryceij, T. (1990) Gibbs sampling in Bayesian networks. Artificial Intelligence, 46, 351–63.

    Google Scholar 

  • Jensen, F. (1994) Implementation aspects of various propagation algorithms in Hugin. Technical Report R 94–2014, University of Aalborg, Denmark.

    Google Scholar 

  • Jensen, F. V. (1996) An Introduction to Bayesian Networks. UCL Press, London.

    Google Scholar 

  • Jensen, C. S., Kong, A. and Kjærulff, U. (1993) Blocking Gibbs sampling in very large probabilistic expert systems. Technical Report R 93–2031, University of Aalborg, Denmark.

    Google Scholar 

  • Jog, P., Suh, J. Y. and Van Gucht, D. (1989) The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 110–15.

  • Kjærulff, U. (1990) Triangulation of graphs-Algorithms giving small total state space. Technical Report R 90–09, University of Aalborg, Denmark.

    Google Scholar 

  • Kjærulff, U. (1992) Optimal decomposition of probabilistic networks by simulated annealing. Statistics and Computing, 2, 7–17.

    Google Scholar 

  • Larrañaga, P., Graña, M., D'Anjou, A. and Torrealdea, F. J. (1993) Genetics algorithms elitist probabilistic of degree 1, a generalization of simulated annealing. In Proceedings of the Third Congress of the Italian Association for Artificial Intelligence, IA*AI '93, Torino, Italy, pp. 208–17.

  • Larrañaga, P., Kuijpers, C. M. H., Poza, M. and Murga, R. H. (1994) Optimal decomposition of Bayesian networks by genetic algorithms. Internal Report EHU-KZAA-IKT–3–94, University of the Basque Country, Spain.

    Google Scholar 

  • Larrañaga, P., Kuijpers, C. M. H., Murga, R. H. and Yurramendi, Y. (1996a) Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics. Vol. 26, No. 4, pp. 487–93.

    Google Scholar 

  • Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R. H., and Kuijpers, C. M. H. (1996b) Structure learning of Bayesian network by genetic algorithms: a performance analysis of control parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence (to appear).

  • Larrañaga, P., Murga, R. H., Poza, M. and Kuijpers, C. M. H. (1996c) Structure learning of Bayesian networks by hybrid genetic algorithms. In Learning from Data: Artificial Intelligence and Statistics V, (D. Fisher and H. Lenz, eds), Springer-Verlag, New York, pp. 165–74.

    Google Scholar 

  • Larrañaga, P., Kuijpers, C. M. H. and Murga, R. H. (1996d) >Evolutionary algorithms for the travelling salesman problem: A review of representations and operators. Artificial Intelligence Review (to appear).

  • Lauritzen, S. L. and Spiegelhalter, D. J. (1988) Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society, Series B, 50, 157–224.

    Google Scholar 

  • Lidd, M. L. (1991) The traveling salesman problem domain application of a fundamentally new approach to utilizing genetic algorithms. Technical Report, MITRE Corporation.

  • Michalewicz, Z. (1992) Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, Berlin.

    Google Scholar 

  • Mühlenbein, H. (1989) Parallel genetic algorithms, population genetics and combinatorial optimization. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 416–21.

  • Neapolitan, R. E. (1990) Probabilistic Reasoning in Expert Systems, Theory and Algorithms. Wiley, New York.

    Google Scholar 

  • Oliver, I. M., Smith, D. J. and Holland, J. R. C. (1987) A study of permutation crossover operators on the TSP. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 224–30.

    Google Scholar 

  • Pearl, J. (1986) Fusion, propagation, and structuring in belief networks. Artificial Intelligence, 29, 241–88.

    Google Scholar 

  • Pearl, J. (1987) Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence, 32, 245–57.

    Google Scholar 

  • Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, Palo Alto, CA.

    Google Scholar 

  • Rose, D. J., Tarjan, R. E. and Lueker, G. S. (1976) Algorithmic aspects of vertex elimination on graphs. SIAM Journal on Computing, 5, 266–83.

    Google Scholar 

  • Rudolph, G. (1994) Convergence analysis of canonical genetic algorithms. IEEETransactions on Neural Networks, 5, 96–101.

    Google Scholar 

  • Seniw, D. (1991) A genetic algorithm for the traveling salesman problem. M.Sc. Thesis, University of North Carolina at Charlotte.

  • Shachter, R. D. (1988) Probabilistic inference and influence diagrams. Operations Research, 36, 589–604.

    Google Scholar 

  • Shachter, R. D. and Peot, M. A. (1990) Simulation approaches to general probabilistic inference on belief networks. In Uncertainty in Artificial Intelligence 5 (M. Henrion, R. D. Shachter, L. N. Kanal and J. F. Lemmer, eds), pp. 221–34. Elsevier, Amsterdam.

    Google Scholar 

  • Shwe, M. and Cooper, G. (1991) An empirical analysis of likelihood-weighting simulation on a large, multiply connected medical belief network. Computers and Biomedical Research, 24, 453–75.

    Google Scholar 

  • Suh, J. Y. and Van Gucht, D. (1987) Incorporating heuristic information into genetic search. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 100–7.

  • Syswerda, G. (1991) Schedule optimization using genetic algorithms. In Handbook of Genetic Algorithms, (L. Davis, ed.), pp. 332–49. Van Nostrand Reinhold, New York.

    Google Scholar 

  • Tarjan, R. E. and Yannakakis, M. (1984) Simple linear-time algorithms to test chordality of graphs, test acyclicity of hypergraphs, and selectively reduce acyclic hypergraphs. SIAM Journal on Computing, 13, 566–79.

    Google Scholar 

  • Wen, W. X. (1990) Decomposing belief networks by simulated annealing. In (C. P. Tsang, ed.) Proceedings of the Australian 1990 Joint Conference on Artificial Intelligence, pp. 103–18. World Scientific Publishers, Perth, WA.

    Google Scholar 

  • Wen, W. X. (1991) Optimal decomposition of belief networks. In Uncertainty in Artificial Intelligence 6 (P. P. Bonissone, M. Henrion, L. N. Kanal, and J. F. Lemmer, eds), pp. 209–24. North-Holland, Amsterdam.

  • Whitley, D. (1989) The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 116–21.

  • Whitley D. and Hanson, T. (1987) Optimizing neural networks using faster, more accurate genetic search. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA, pp. 391–6.

  • Whitley, D., Starkweather, T. and Fuquay, D. (1989) Scheduling problems and travelling salesman: The genetic edge recombination operator. In Proceedings on the Third International Conference on Genetic Algorithms, Arlington, VA, pp. 133–40.

  • Whitley, D., Starkweather, T. and Shaner, D. (1991) The traveling salesman and sequence scheduling: Quality solutions using genetic edge recombination. In Handbook of Genetic Algorithms (L. Davis, ed), pp. 350–72. Van Nostrand Reinhold, New York.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Artificial Intelligence, University of the Basque Country, PO Box 649, 20080, San Sebastia´n, Spain

    Pedro Larrañaga, Mikel Poza & Roberto H. Murga

  2. Department of Applied Mathematics, University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands

    Cindy M. H. Kuijpers

Authors
  1. Pedro Larrañaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cindy M. H. Kuijpers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mikel Poza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto H. Murga
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Larrañaga, P., Kuijpers, C.M.H., Poza, M. et al. Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms. Statistics and Computing 7, 19–34 (1997). https://doi.org/10.1023/A:1018553211613

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018553211613

Search

Navigation