Skip to main content
SpringerLink
Log in

An entropy-based self-adaptive simulated annealing

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

Simulated annealing (SA) algorithms are capable of solving discrete and continuous problems. However, they are less efficient than other algorithms in solving applied problems because of their dependency on controlling parameters definition method. In the current work, a self-adaptive simulated annealing (SASA) method is presented based on entropy concept and thermodynamic laws in order to optimize the setting parameters. To provide a dynamic cooling rate and Markov chain length with a comparative relation to problem conditions, the proposed schedule utilizes thermodynamic concepts of entropy and ensemble average energy. In the proposed algorithm, simulation of the atomic motion is implemented based on velocity definition in thermodynamics and time definition in probabilistic processes. The SASA is evaluated by CEC2015 problem and compared with three other adaptive simulated annealing algorithms, a standard SA and four other metaheuristic methods using three different comparison criteria: Wilcoxon test, median, mean and standard deviation. The SASA has shown satisfactory outcomes in most unimodal, multimodal, and hybrid functions in CEC2015. It has proved to be more explorative, has obtained far better solutions, and has showed the best convergence speed compared with other algorithms when engaged in exploitation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Notes

  1. \(N_{a}\) is the number of acceptances and \(N_{t}\) is the number of iterations in each inner loop.

  2. \(\Delta T\) is the parameter for temperature reduction.

  3. \(\sigma\left(T\right)\) standard deviation of the energy (cost function) of the points in the Markov chain at T & δ = 0.1 is called the distance parameter.

  4. U is some upper bound on \(\left( {E\left( s \right) - E_{ \text{min} } } \right)\) and \(\gamma\) a small real number.

  5. \(E_{ \text{min} }^{k}\) is minimal energy at current temperature, \(E_{\text{avg}}^{k}\) is average energy of attempted moves at current temperature.

  6. \(\sigma \left( {T_{k} } \right)\) is standard deviation of the expected energy (cost function) of the points in the Markov chain at \(T_{k}\) and \(\Delta \left( {T_{k} } \right) = \sigma \left( {T_{k} } \right)/\mu\), by \(\mu \in \left[ {1,20} \right]\).

  7. \(\alpha\) is a constant equal to temperature when change in energy is zero and \(\varGamma\) is minimum energy (cost function) achieved in any iteration + a fixed constant C.

  8. \(U^{j}\) and \(L^{j}\) are the lower and upper bounds of the variable and \(\rho\) is a uniform random number from [0, 1].

  9. \(\rho\) is a uniform random number from [0, 1], N is the maximal generation number, and b = 5 is a system parameter determining the degree of dependency on iteration number and \(\eta\) is randomly generated.

  10. The number of accepted moves along the i-axis since the last step adjustment is \(n_{i}\). \(N_{\text{s}} = 20 \;{\text{and}} \;c_{i} = 2\). And r is a random number generated in the range [-1, l] \(e_{i}\) is the vector of the ith coordinate direction; \(d_{i + 1}^{j}\) is the component of the step vector d.

References

  1. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680. https://doi.org/10.1126/science.220.4598.671

    Article  MathSciNet  MATH  Google Scholar 

  2. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092. https://doi.org/10.1063/1.1699114

    Article  MATH  Google Scholar 

  3. Černý V (1985) Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45(1):41–51. https://doi.org/10.1007/BF00940812

    Article  MathSciNet  MATH  Google Scholar 

  4. Rose J, Klebsch W, Wolf Jr (1990) Temperature measurement and equilibrium dynamics of simulated annealing placements. IEEE Trans Comput Aided Des Integr Circuits Syst 9(3):253–259. https://doi.org/10.1109/43.46801

    Article  Google Scholar 

  5. Greene JW, Supowit KJ (1986) Simulated annealing without rejected moves. IEEE Trans Comput Aided Des Integr Circuits Syst 5(1):221–228. https://doi.org/10.1109/TCAD.1986.1270190

    Article  Google Scholar 

  6. Aluffi-Pentini F, Parisi V, Zirilli F (1985) Global optimization and stochastic differential equations. J Optim Theory Appl 47(1):1–16. https://doi.org/10.1007/BF00941312

    Article  MathSciNet  MATH  Google Scholar 

  7. Geman S, Geman D (1984) Stochastic relaxation, gibbs distribution, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell Pami-6 6:721–741

    Article  Google Scholar 

  8. Szu H, Hartley R (1987) Fast simulated annealing. Phys Lett A 122(3–4):157–162. https://doi.org/10.1016/0375-9601(87)90796-1

    Article  Google Scholar 

  9. Shieh H-L, Kuo C-C, Chiang C-M (2011) Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification. Appl Math Comput 218(8):4365–4383. https://doi.org/10.1016/j.amc.2011.10.012

    Article  MATH  Google Scholar 

  10. Siarry P, Berthiau G, Durdin F, Haussy J (1997) Enhanced simulated annealing for globally minimizing functions of many-continuous variables. ACM Trans Math Softw 23(2):209–228. https://doi.org/10.1145/264029.264043

    Article  MathSciNet  MATH  Google Scholar 

  11. Ali MM, Gabere MN, Zhu W (2012) A derivative-free variant called DFSA of Dekkers and Aarts’ continuous simulated annealing algorithm. Appl Math Comput 219(2):605–616. https://doi.org/10.1016/j.amc.2012.06.050

    Article  MathSciNet  MATH  Google Scholar 

  12. Xinchao Z (2011) Simulated annealing algorithm with adaptive neighborhood. Appl Soft Comput 11(2):1827–1836. https://doi.org/10.1016/j.asoc.2010.05.029

    Article  Google Scholar 

  13. Xavier-De-Souza S, Suykens JAK, Vandewalle J, Ds Bolle (2010) Coupled simulated annealing. IEEE Trans Syst Man Cybern B Cybern 40(2):320–335. https://doi.org/10.1109/TSMCB.2009.2020435

    Article  Google Scholar 

  14. Tsallis C, Da Stariolo (1996) Generalized simulated annealing. Phys A 233:395–406

    Article  Google Scholar 

  15. Bertsimas D, Nohadani O (2010) Robust optimization with simulated annealing. J Glob Optim 48(2):323–334. https://doi.org/10.1007/s10898-009-9496-x

    Article  MathSciNet  MATH  Google Scholar 

  16. Wang PP, Chen D-S (1996) Continuous optimization by a variant of simulated annealing. Comput Optim Appl 6:59–71. https://doi.org/10.1007/BF00248009

    Article  MathSciNet  MATH  Google Scholar 

  17. Ingber L (1989) Very fast simulated re-annealing. Math Comput Model 12(8):967–973. https://doi.org/10.1016/0895-7177(89)90202-1

    Article  MathSciNet  MATH  Google Scholar 

  18. Dekkers A, Aarts E (1991) Global optimization and simulated annealing. Math Program 50(1–3):367–393. https://doi.org/10.1007/BF01594945

    Article  MathSciNet  MATH  Google Scholar 

  19. Triki E, Collette Y, Siarry P (2005) A theoretical study on the behavior of simulated annealing leading to a new cooling schedule. Eur J Oper Res 166(1):77–92. https://doi.org/10.1016/j.ejor.2004.03.035

    Article  MathSciNet  MATH  Google Scholar 

  20. de Vicente J, Lanchares J, Hermida R (2003) Placement by thermodynamic simulated annealing. Phys Lett A 317(5–6):415–423. https://doi.org/10.1016/j.physleta.2003.08.070

    Article  MATH  Google Scholar 

  21. Corana A, Marchesi M, Martini C, Ridella S (1987) Minimizing multimodal functions of continuous variables with the ‘simulated annealing’ algorithm. ACM Trans Math Softw 13(3):262–280. https://doi.org/10.1145/66888.356281

    Article  MathSciNet  MATH  Google Scholar 

  22. Romeijn HE, Smith RL (1994) Simulated annealing for constrained global optimization. J Glob Optim 5(2):101–126. https://doi.org/10.1007/BF01100688

    Article  MathSciNet  MATH  Google Scholar 

  23. Salazar R, Toral R (1997) Simulated annealing using hybrid Monte Carlo. J Stat Phys 89(1):1047–1060. https://doi.org/10.1007/BF02764221

    Article  MATH  Google Scholar 

  24. Andricioaei I, Straub J (1996) Generalized simulated annealing algorithms using Tsallis statistics: application to conformational optimization of a tetrapeptide. Phys Rev E 53(4):R3055–R3058. https://doi.org/10.1103/PhysRevE.53.R3055

    Article  Google Scholar 

  25. Xin Y, Guojie L (1991) General simulated annealing. J Comput Sci Technol 6(4):328–329

    MathSciNet  Google Scholar 

  26. Shojaee GK, Shakouri GH, Menhaj MB (2011) A mushy state simulated annealing. AUT J Model Simul 43(2):59–70. https://doi.org/10.22060/MISCJ.2011.174

    Article  Google Scholar 

  27. Haines LM (1987) The application of the annealing algorithm to the construction of exact optimal designs for linear—regression models. Technometrics 29(4):439–447. https://doi.org/10.1080/00401706.1987.10488272

    Article  MATH  Google Scholar 

  28. Lundy M, Mees A (1986) Convergence of an annealing algorithm. Math Program 34(1):111–124. https://doi.org/10.1007/BF01582166

    Article  MathSciNet  MATH  Google Scholar 

  29. Locatelli M (2000) Convergence of the simulated annealing algorithm for continuous global optimization. J Glob Optim 18:219–234

    Article  MathSciNet  Google Scholar 

  30. Bohachevsky IO, Johnson ME, Stein ML (1986) Generalized simulated annealing for function optimization. Technometrics 28(3):209–217. https://doi.org/10.1080/00401706.1986.10488128

    Article  MATH  Google Scholar 

  31. Shen Y, Kiatsupaibul S, Zabinsky ZB, Smith RL (2007) An analytically derived cooling schedule for simulated annealing. J Glob Optim 38(3):333–365. https://doi.org/10.1007/s10898-006-9068-2

    Article  MathSciNet  MATH  Google Scholar 

  32. Ansari N, Sarasa R, Wang G (1993) An efficient annealing algorithm for global optimization in Boltzmann machines. Appl Intell 3(3):177–192

    Article  Google Scholar 

  33. Zhao X, Zhou Y, Xiang Y (2019) A grouping particle swarm optimizer. Appl Intell. https://doi.org/10.1007/s10489-019-01409-4

    Article  Google Scholar 

  34. Ning Y, Peng Z, Dai Y, Bi D, Wang J (2018) Enhanced particle swarm optimization with multi-swarm and multi-velocity for optimizing high-dimensional problems. Appl Intell 49(2):335–351. https://doi.org/10.1007/s10489-018-1258-3

    Article  Google Scholar 

  35. Rashedi E, Nezamabadi-pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248. https://doi.org/10.1016/j.ins.2009.03.004

    Article  MATH  Google Scholar 

  36. Tian L, Li Z, Yan X (2018) Differential evolution algorithm directed by individual difference information between generations and current individual information. Appl Intell 49(2):628–649. https://doi.org/10.1007/s10489-018-1255-6

    Article  Google Scholar 

  37. Duan M, Yang H, Liu H, Chen J (2018) A differential evolution algorithm with dual preferred learning mutation. Appl Intell 49(2):605–627. https://doi.org/10.1007/s10489-018-1267-2

    Article  Google Scholar 

  38. Gelfand SB, Mitter SK (1991) Simulated annealing type algorithms for multivariate optimization. Algorithmica 6:419–436

    Article  MathSciNet  Google Scholar 

  39. Gelfand SB, Mitter SK (1991) Weak convergence of markov and algorithms to sampling diffusions. J Optim Theory Appl 68(3):483–498

    Article  MathSciNet  Google Scholar 

  40. Pavlyukevich I (2007) Lévy flights, non-local search and simulated annealing. J Comput Phys 226(2):1830–1844. https://doi.org/10.1016/j.jcp.2007.06.008

    Article  MathSciNet  MATH  Google Scholar 

  41. Reif F, Sevenich RA (1971) Statistical physics (Berkeley physics course—volume 5). Am J Phys 39(2):235. https://doi.org/10.1119/1.1986109

    Article  Google Scholar 

  42. Szu HH, Hartley RL (1987) Nonconvex optimization by fast simulated annealing. Proc IEEE 75(11):1538–1540. https://doi.org/10.1109/PROC.1987.13916

    Article  Google Scholar 

  43. Liang JJ, Qu BY, Suganthan PN, Chen Q (2014) Problem definitions and evaluation criteria for the CEC 2015 competition on learning-based real-parameter single objective optimization. Technical report, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Technical Report, Nanyang Technological University, Singapore

  44. Arora J, Elwakeil O, Chahande A, Hsieh C (1995) Global optimization methods for engineering applications: a review. Struct Optim 9(3–4):137–159

    Article  Google Scholar 

  45. Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657. https://doi.org/10.1109/TEVC.2006.872133

    Article  Google Scholar 

  46. Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195. https://doi.org/10.1162/106365601750190398

    Article  Google Scholar 

  47. Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295. https://doi.org/10.1109/tevc.2005.857610

    Article  Google Scholar 

  48. Garcia-Martinez C, Lozano M, Herrera F, Molina D, Sanchez AM (2008) Global and local real-coded genetic algorithms based on parent-centric crossover operators. Eur J Oper Res 185(3):1088–1113. https://doi.org/10.1016/j.ejor.2006.06.043

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

    Kambiz Shojaee Ghandeshtani & Habib Rajabi Mashhadi

  2. Department of Electrical Engineering, Faculty of Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

    Habib Rajabi Mashhadi

  3. Center of Excellence on Soft Computing and Intelligent Information Processing (SCIIP), Ferdowsi University of Mashhad, Mashhad, Iran

    Kambiz Shojaee Ghandeshtani & Habib Rajabi Mashhadi

Authors
  1. Kambiz Shojaee Ghandeshtani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Habib Rajabi Mashhadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Habib Rajabi Mashhadi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shojaee Ghandeshtani, K., Mashhadi, H.R. An entropy-based self-adaptive simulated annealing. Engineering with Computers 37, 1329–1355 (2021). https://doi.org/10.1007/s00366-019-00887-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-019-00887-x

Keywords

Search

Navigation