Skip to main content
SpringerLink
Log in

Hierarchical Bayesian Optimization Algorithm

  • Chapter
Book cover Scalable Optimization via Probabilistic Modeling

Part of the book series: Studies in Computational Intelligence ((SCI,volume 33))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arst, R. E. (2002). Which are better, probabilistic model-building genetic algorithms (PMBGAs) or simple genetic algorithms (sGAs)? A com-parison for an optimal groundwater remediation design problem. PhD thesis, University of Illinois at Urbana-Champaign, Department of Civil Engineering, Urbana, IL

    Google Scholar 

  2. Asoh, H., and Mühlenbein, H. (1994). On the mean convergence time of evolutionary algorithms without selection and mutation. Parallel Problem Solving from Nature, pp. 88-97

    Google Scholar 

  3. Baluja, S. (1994). Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Tech. Rep. No. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA

    Google Scholar 

  4. Baluja, S. (2002). Using a priori knowledge to create probabilistic models for optimization. International Journal of Approximate Reasoning, 31(3):193-220

    Article  MATH  MathSciNet  Google Scholar 

  5. Bosman, P. A. N., and Thierens, D. (1999). Linkage information process-ing in distribution estimation algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-99), I:60-67

    Google Scholar 

  6. Butz, M. V., Pelikan, M., Llorà, X., and Goldberg, D. E. (2005). Extracted global structure makes local building block processing effective in XCS. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2005), 1:655-662

    Article  Google Scholar 

  7. Cantú-Paz, E. (2000). Efficient and Accurate Parallel Genetic Algorithms. Kluwer, Boston, MA

    MATH  Google Scholar 

  8. Chickering, D. M., Heckerman, D., and Meek, C. (1997). A Bayesian approach to learning Bayesian networks with local structure. Technical Report MSR-TR-97-07, Microsoft Research, Redmond, WA

    Google Scholar 

  9. Cooper, G. F., and Herskovits, E. H. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9:309-347

    MATH  Google Scholar 

  10. Deb, K., and Goldberg, D. E. (1994). Sufficient conditions for deceptive and easy binary functions. Annals of Mathematics and Artificial Intelligence, 10:385-408

    Article  MATH  MathSciNet  Google Scholar 

  11. Etxeberria, R., and Larrañaga, P. (1999). Global optimization using Bayesian networks. In Rodriguez, A. A. O., Ortiz, M. R. S., and Hermida, R. S., (Eds.), Second Symposium on Artificial Intelligence (CIMAF-99), pp. 332-339, Habana, Cuba. Institude of Cybernetics, Mathematics, and Physics and Ministry of Science, Technology and Environment

    Google Scholar 

  12. Friedman, N., and Goldszmidt, M. (1999). Learning Bayesian networks with local structure. In Jordan, M. I., (Ed.), Graphical models, pp. 421-459. MIT, Cambridge, MA

    Google Scholar 

  13. Friedman, N., and Yakhini, Z. (1996). On the sample complexity of learning Bayesian networks. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-96), pp. 274-282

    Google Scholar 

  14. Goldberg, D. E. (1994). First flights at genetic-algorithm Kitty Hawk. IlliGAL Report No. 94008, University of Illinois at Urbana-Champaign, Urbana, IL

    Google Scholar 

  15. Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms, volume 7 of Genetic Algorithms and Evolutionary Computation. Kluwer, Boston, MA

    Google Scholar 

  16. Goldberg, D. E., Deb, K., and Clark, J. H. (1992). Genetic algorithms, noise, and the sizing of populations. Complex Systems, 6:333-362

    MATH  Google Scholar 

  17. Goldberg, D. E., Sastry, K., and Latoza, T. (2001). On the supply of building blocks. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 336-342

    Google Scholar 

  18. Harik, G. (1999). Linkage learning via probabilistic modeling in the ECGA. IlliGAL Report No. 99010, University of Illinois at UrbanaChampaign, Illinois Genetic Algorithms Laboratory, Urbana, IL

    Google Scholar 

  19. Harik, G., and Lobo, F. (1999). A parameter-less genetic algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-99), I:258-265

    Google Scholar 

  20. Harik, G. R. (1995). Finding multimodal solutions using restricted tournament selection. Proceedings of the International Conference on Genetic Algorithms (ICGA-95), pp. 24-31

    Google Scholar 

  21. Harik, G. R., Cantú-Paz, E., Goldberg, D. E., and Miller, B. L. (1997). The gambler’s ruin problem, genetic algorithms, and the sizing of pop-ulations. Proceedings of the International Conference on Evolutionary Computation (ICEC-97), pp. 7-12. Also IlliGAL Report No. 96004

    Google Scholar 

  22. Heckerman, D., Geiger, D., and Chickering, D. M. (1994). Learning Bayesian networks: The combination of knowledge and statistical data. Technical Report MSR-TR-94-09, Microsoft Research, Redmond, WA

    Google Scholar 

  23. Henrion, M. (1988). Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Lemmer, J. F., and Kanal, L. N., (Eds.), Uncertainty in Artificial Intelligence, pp. 149-163. Elsevier, Amsterdam London New York

    Google Scholar 

  24. Holland, J. H.(1975). Adaptation in natural and artificial systems. University of Michigan, Ann Arbor, MI

    Google Scholar 

  25. Howard, R. A., and Matheson, J. E. (1981). Influence diagrams. In Howard, R. A., and Matheson, J. E., (Eds.), Readings on the principles and applications of decision analysis, volume II, pp. 721-762. Strategic Decisions Group, Menlo Park, CA

    Google Scholar 

  26. Khan, N. (2003). Bayesian optimization algorithms for multiobjective and hierarchically difficult problems. Master’s thesis, University of Illinois at Urbana-Champaign, Urbana, IL

    Google Scholar 

  27. Larrañaga, P., and Lozano, J. A., (Eds.) (2002). Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer, Boston, MA

    MATH  Google Scholar 

  28. Laumanns, M., and Ocenasek, J. (2002). Bayesian optimization algorithms for multi-objective optimization. Parallel Problem Solving from Nature, pp. 298-307

    Google Scholar 

  29. Li, J., and Aickelin, U. (2003). A Bayesian optimization algorithm for the nurse scheduling problem. Proceedings of the IEEE Congress on Evo-lutionary Computation 2003 (CEC-2003), pp. 2149-2156

    Google Scholar 

  30. Lobo, F. G., Goldberg, D. E., and Pelikan, M. (2000). Time complexity of genetic algorithms on exponentially scaled problems. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 151-158. Also IlliGAL Report No. 2000016

    Google Scholar 

  31. Looks, M., Goertzel, B., and Pennachin, C.(2005). Learning computer programs with the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2005), 1:747-748

    Article  Google Scholar 

  32. Looks, M., Groetzel, B., and Pennachin, C. (2003). Learning computer programs with the Bayesian optimization algorithm. Technical report, Object Sciences Corporation and Novamente LLC

    Google Scholar 

  33. Mühlenbein, H. (1992a). How genetic algorithms really work: I.Mutation and Hillclimbing. Parallel Problem Solving from Nature, pp. 15-25

    Google Scholar 

  34. Mühlenbein, H. (1992b). How genetic algorithms really work: I.Mutation and Hillclimbing. In Männer, R., and Manderick, B., (Eds.), Parallel Problem Solving from Nature, pp. 15-25, Elsevier, Amsterdam

    Google Scholar 

  35. Mühlenbein, H., and Mahnig, T. (2002). Evolutionary optimization and the estimation of search distributions with aplication to graph bipartitioning. International Journal of Approximate Reasoning, 31(3):157-192

    Article  MATH  MathSciNet  Google Scholar 

  36. Mühlenbein, H., and Paass, G. (1996). From recombination of genes to the estimation of distributions I. Binary parameters. In Eiben, A., Bäck, T., Shoenauer, M., and Schwefel, H., (Eds.), Parallel Problem Solving from Nature, pp. 178-187, Springer, Berlin Heidelberg New York

    Google Scholar 

  37. Mühlenbein, H., and Schlierkamp-Voosen, D. (1993). Predictive models for the breeder genetic algorithm: I. Continuous parameter optimization. Evolutionary Computation, 1(1):25-49

    Article  Google Scholar 

  38. Ocenasek, J., and Schwarz, J. (2000). The parallel Bayesian optimization algorithm. In Proceedings of the European Symposium on Computational Inteligence, pp. 61-67. Physica, Wurzburg (Wien)

    Google Scholar 

  39. Ocenasek, J., and Schwarz, J. (2002). Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In 2nd Euro-International Symposium on Computational Intelligence, pp. 227-232

    Google Scholar 

  40. Ocenasek, J., Schwarz, J., and Pelikan, M. (2003). Design of multithreaded estimation of distribution algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003), pp. 1247-1258

    Google Scholar 

  41. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Mateo, CA

    Google Scholar 

  42. Pelikan, M. (2002). Bayesian optimization algorithm: From single level to hierarchy. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL. Also IlliGAL Report No. 2002023

    Google Scholar 

  43. Pelikan, M.(2005). Hierarchical Bayesian optimization algorithm: Toward a new generation of evolutionary algorithms. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  44. Pelikan, M., and Goldberg, D. E. (2000). Hierarchical problem solving by the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 275-282

    Google Scholar 

  45. Pelikan, M., and Goldberg, D. E. (2001). Escaping hierarchical traps with competent genetic algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 511-518

    Google Scholar 

  46. Pelikan, M., and Goldberg, D. E. (2003a). Hierarchical BOA solves Ising spin glasses and MAXSAT. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003), II:1275-1286

    Google Scholar 

  47. Pelikan, M., and Goldberg, D. E. (2003b). A hierarchy machine: Learning to optimize from nature and humans. Complexity, 8(5):36-45

    Article  Google Scholar 

  48. Pelikan, M., Goldberg, D. E., and Cantú-Paz, E. (1999). BOA: The Bayesian optimization algorithm. Proceedings of the Genetic and Evo-lutionary Computation Conference (GECCO-99), I:525-532

    Google Scholar 

  49. Pelikan, M., Goldberg, D. E., and Cantú-Paz, E. (2000). Linkage problem, distribution estimation, and Bayesian networks. Evolutionary Computation, 8(3):311-341. Also IlliGAL Report No. 98013

    Article  Google Scholar 

  50. Pelikan, M., Goldberg, D. E., and Lobo, F. (2002a). A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1):5-20

    Article  MATH  MathSciNet  Google Scholar 

  51. Pelikan, M., Goldberg, D. E., and Sastry, K. (2001). Bayesian optimization algorithm, decision graphs, and Occam’s razor. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 519-526. Also IlliGAL Report No. 2000020

    Google Scholar 

  52. Pelikan, M., and Lin, T.-K. (2004). Parameter-less hierarchical BOA. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:24-35

    Google Scholar 

  53. Pelikan, M., Ocenasek, J., Trebst, S., Troyer, M., and Alet, F. (2004). Computational complexity and simulation of rare events of Ising spin glasses. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:36-47

    Google Scholar 

  54. Pelikan, M., and Sastry, K. (2004). Fitness inheritance in the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:48-59

    Google Scholar 

  55. Pelikan, M., Sastry, K., and Goldberg, D. E. (2002b). Scalability of the Bayesian optimization algorithm. International Journal of Approximate Reasoning, 31(3):221-258

    Article  MATH  MathSciNet  Google Scholar 

  56. Pelikan, M., Sastry, K., and Goldberg, D. E. (2005a). Multiobjective hBOA, clustering, and scalability. Proceedings of the Genetic and Evolu-tionary Computation Conference (GECCO-2005), pp. 663-670

    Google Scholar 

  57. Pelikan, M., Sastry, K., and Goldberg, D. E. (2005b). Sporadic model building for efficiency enhancement of hBOA. Technical report, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory

    Google Scholar 

  58. Pelikan, M., Goldberg, D. E., and Tsutsui, S. (2003). Getting the best of both worlds: Discrete and continuous genetic and evolutionary algorithms in concert. Information Sciences, 156 (3-4), 147-171

    Article  MathSciNet  Google Scholar 

  59. Rissanen, J. J. (1978). Modelling by shortest data description. Automatica, 14:465-471

    Article  MATH  Google Scholar 

  60. Rissanen, J. J. (1989). Stochastic complexity in statistical inquiry. World Scientific, Singapore

    MATH  Google Scholar 

  61. Rissanen, J. J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42(1):40-47

    Article  MATH  MathSciNet  Google Scholar 

  62. Rothlauf, F. (2001). Towards a theory of representations for genetic and evolutionary algorithms: Development of basic concepts and their application to binary and tree representations. PhD thesis, University of Bayreuth, Beyreuth, Germany

    Google Scholar 

  63. Rothlauf, F., Goldberg, D. E., and Heinzl, A. (2000). Bad codings and the utility of well-designed genetic algorithms. IlliGAL Report No. 200007, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL

    Google Scholar 

  64. Santarelli, S., Goldberg, D. E., and Yu, T.-L. (2004). Optimization of a constrained feed network for an antenna array using simple and compe-tent genetic algorithm techniques. Proceedings of the Workshop Military and Security Application of Evolutionary Computation (MSAEC-2004)

    Google Scholar 

  65. Sastry, K. (2001). Efficient atomic cluster optimization using a hybrid extended compact genetic algorithm with seeded population. IlliGAL Re-port No. 2001018, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL

    Google Scholar 

  66. Sastry, K., Pelikan, M., and Goldberg, D. E. (2004). Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. Proceedings of the IEEE Conference on Evolutionary Computation, pp. 720-727

    Google Scholar 

  67. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6:461-464

    Article  MATH  MathSciNet  Google Scholar 

  68. Schwarz, J., and Ocenasek, J. (1999). Experimental study: Hypergraph partitioning based on the simple and advanced algorithms BMDA and BOA. In Proceedings of the International Conference on Soft Computing, pp. 124-130, Brno, Czech Republic. PC-DIR

    Google Scholar 

  69. Schwarz, J., and Ocenasek, J. (2000). A problem-knowledge based evolutionary algorithm KBOA for hypergraph partitioning. In Proceedings of the Fourth Joint Conference on Knowledge-Based Software Engineering, pp. 51-58, IO, Brno, Czech Republic

    Google Scholar 

  70. Simon, H. A. (1968). The Sciences of the Artificial. MIT, Cambridge, MA

    Google Scholar 

  71. Thierens, D. (1995). Analysis and design of genetic algorithms. PhD thesis, Katholieke Universiteit Leuven, Leuven, Belgium

    Google Scholar 

  72. Thierens, D., and Goldberg, D. (1994). Convergence models of genetic algorithm selection schemes. Parallel Problem Solving from Nature, pp. 116-121

    Google Scholar 

  73. Thierens, D., Goldberg, D. E., and Pereira, A. G. (1998). Domino convergence, drift, and the temporal-salience structure of problems. Proceedings of the International Conference on Evolutionary Computation (ICEC-98), pp. 535-540

    Google Scholar 

  74. Watson, R. A., Hornby, G. S., and Pollack, J. B. (1998). Modeling building-block interdependency. Parallel Problem Solving from Nature, pp. 97-106

    Google Scholar 

  75. Wilson, S. W. (1995). Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149-175

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Missouri at St. Louis, One University Blvd. St. Louis, MO 63121, USA

    Dr. Martin Pelikan

  2. University of Illinois at Urbana-Champaign 104 S., Mathews Avenue Urbana, IL 61801, USA

    David E. Goldberg

Authors
  1. Dr. Martin Pelikan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David E. Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Math. And Computer Science, University of Missouri at St. Louis, One University Blvd., 320 CCB, MO 63121, St. Louis, USA

    Martin Pelikan

  2. Department of Industrial and Enterprise Systems Engineering, Illinois Genetic Algorithms Laboratory, 414 TB, 104 S. Mathews Ave., IL 61801, Urbana, USA

    Kumara Sastry

  3. Yahoo! Inc., 701 First Avenue, CA 94809, Sunnyvale, USA

    Erick CantúPaz

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Pelikan, M., Goldberg, D.E. (2006). Hierarchical Bayesian Optimization Algorithm. In: Pelikan, M., Sastry, K., CantúPaz, E. (eds) Scalable Optimization via Probabilistic Modeling. Studies in Computational Intelligence, vol 33. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-34954-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-34954-9_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34953-2

  • Online ISBN: 978-3-540-34954-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation