Skip to main content
Springer Nature Link
Log in

Evolving Smoothing Kernels for Global Optimization

  • Conference paper
  • First Online:
Applications of Evolutionary Computation (EvoApplications 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9598))

Included in the following conference series:

  • 2872 Accesses

Abstract

The Diffusion-Equation Method (DEM) – sometimes synonymously called the Continuation Method – is a well-known natural computation approach in optimization. The DEM continuously transforms the objective function by a (Gaussian) kernel technique to reduce barriers separating local and global minima. Now, the DEM can successfully solve problems of small sizes. Here, we present a generalization of the DEM to use convex combinations of smoothing kernels in Fourier space. We use a genetic algorithm to incrementally optimize the (meta-)combinatorial problem of finding better performing kernels for later optimization of an objective function. For two test applications we derive and show their transferability to larger problems. Most strikingly, the original DEM failed on a number of the test instances to find the global optimum while our transferable kernels – obtained via evolutionary computations – were able to find the global optimum.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    For example, the DEM fails to converge to the global optimum of \(g(x) = \cos (x-3.4) + 6\cos (2x + 1) + 3\cos (3x+2.5)\) for all starting points.

  2. 2.

    For \(\alpha \)’s the width of the normal distribution was set to 0.2 and for the \(\nu \)’s to 1.0.

  3. 3.

    Here degrees of freedom = \(3(N-2)\) due to a free choice of the point of origin and orientation, thus translational and rotational invariance of the potential V. The prefactor 3 is the relation between no. of particles and no. of Eucledian coordinates.

References

  1. Piela, L., Kostrowicki, J., Scheraga, H.A.: On the multiple-minima problem in the conformational analysis of molecules: deformation of the potential energy hypersurface by the diffusion equation method. J. Phys. Chem. 93(8), 3339–3346 (1989)

    Article  Google Scholar 

  2. Wawak, R.J., Pillardy, J., Liwo, A., Gibson, K.D., Scheraga, H.A.: Diffusion equation and distance scaling methods of global optimization. J. Phys. Chem. A 102(17), 2904 (1998)

    Article  Google Scholar 

  3. Mobahi, H., Fisher III, J.W.: On the link between Gaussian homotopy continuation and convex envelopes. In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 43–56. Springer, Heidelberg (2015)

    Google Scholar 

  4. Mobahi, H., Fisher III, J.W.: A theoretical analysis of optimization by Gaussian continuation. In: AAAI 2015: 29th Conference on Artificial Intelligence (2015)

    Google Scholar 

  5. Kostrowicki, J., Piela, L., Cherayil, B.J., Scheraga, H.A.: Performance of the diffusion equation method in searches for optimum structures of clusters of Lennard-Jones atoms. J. Phys. Chem. 95(10), 4113–4119 (1991)

    Article  Google Scholar 

  6. Wawak, R., Wimmer, M., Scheraga, H.: Application of the diffusion equation method of global optimization to water clusters. J. Phys. Chem. 96(12), 5138–5145 (1992)

    Article  Google Scholar 

  7. Hamacher, K.: Optimization of high-dimensional functions with respect to protein-structure-prediction. Dipl. thesis, Dortmund University (1998)

    Google Scholar 

  8. Galassi, M.: GNU Scientific Library Reference Manual. Network Theory Ltd. (2009)

    Google Scholar 

  9. Miklavcic, M.: Applied Functional Analysis and PDEs. World Scientific, Singapore (1998)

    MATH  Google Scholar 

  10. Abels, H.: Pseudodifferential and Singular Integral Operators. Walter de Gruyter, Berlin (2011)

    Book  MATH  Google Scholar 

  11. Marks, R.J.: Handbook of Fourier Analysis & Its Applications. Oxford University Press, Oxford (2009)

    MATH  Google Scholar 

  12. Aluffi-Pentini, F., Parisi, V., Zirilli, F.: Global optimization and stochastic differential equations. J. Optim. Theor. Appl. 47(1), 1–16 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  13. Blum, C., Roli, A.: Hybrid metaheuristics: an introduction. In: Blum, C., José Bleasa Aguilera, M., Roli, A., Sampels, M. (eds.) Hybrid Metaheuristics, pp. 1–30. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  14. Kruse, R., Borgelt, C., Klawonn, F., Moewes, C., Steinbrecher, M., Held, P.: Computational Intelligence: Eine methodische Einführung in Künstliche Neuronale Netze, Evolutionäre Algorithmen, Fuzzy-Systeme und Bayes-Netze. Springer, Heidelberg (2011)

    Book  Google Scholar 

  15. Mitchell, M.: An Introduction to Genetic Algorithms. Bradford, Denver (1998)

    MATH  Google Scholar 

  16. Yu, X., Gen, M.: Decision Engineering. Springer, Heidelberg (2010)

    Google Scholar 

  17. Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Graduate Texts in Mathematics, vol. 194. Springer, Heidelberg (2000)

    MATH  Google Scholar 

  18. Engel, K.J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Semi-group Forum, vol. 63. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  19. Mayer, B.E., Hamacher, K.: Stochastic tunneling transformation during selection in genetic algorithm. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 801–806. ACM, New York (2014)

    Google Scholar 

  20. Nocedal, J., Wright, S.: Numerical Optimization. Operations Research and Financial Engineering. Springer, New York (2006)

    MATH  Google Scholar 

  21. R Development Core Team.: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008)

    Google Scholar 

  22. Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., Weingessel, A.: Misc functions of the Department of Statistics (e1071), TU Wien. R package 1–5 (2008)

    Google Scholar 

  23. Hopfinger, A.: Conformational Properties of Macromolecules. Academic Press, London (1973)

    Google Scholar 

  24. Wales, D.J., Doye, J.P.: Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms. J. Phys. Chem. A 101(28), 5111–5116 (1997)

    Article  Google Scholar 

  25. Wagner, P.: A new constructive proof of the Malgrange-Ehrenpreis theorem. Am. Math. Month. 116(5), 457–462 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Diosan, L., Rogozan, A., Pecuchet, J.P.: Evolving kernel functions for svms by genetic programming. In: Sixth International Conference on Machine Learning and Applications, ICMLA 2007, 19–24 December 2007

    Google Scholar 

  27. Sullivan, K.M., Luke, S.: Evolving kernels for support vector machine classification. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1702–1707. ACM, New York (2007)

    Google Scholar 

Download references

Acknowledgements

KH gratefully acknowledges funding by the LOEWE project compuGene of the Hessen State Ministry of Higher Education, Research and the Arts.

Author information

Authors and Affiliations

  1. Department of Computer Science, TU Darmstadt, Darmstadt, Germany

    Paul Manns & Kay Hamacher

  2. Department of Biology, TU Darmstadt, Darmstadt, Germany

    Kay Hamacher

  3. Department of Physics, TU Darmstadt, Darmstadt, Germany

    Kay Hamacher

Authors
  1. Paul Manns
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kay Hamacher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kay Hamacher .

Editor information

Editors and Affiliations

  1. Politecnico di Torino, Turin, Italy

    Giovanni Squillero

  2. Aalborg University, Copenhagen, Denmark

    Paolo Burelli

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Manns, P., Hamacher, K. (2016). Evolving Smoothing Kernels for Global Optimization. In: Squillero, G., Burelli, P. (eds) Applications of Evolutionary Computation. EvoApplications 2016. Lecture Notes in Computer Science(), vol 9598. Springer, Cham. https://doi.org/10.1007/978-3-319-31153-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31153-1_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31152-4

  • Online ISBN: 978-3-319-31153-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics