Skip to main content
SpringerLink
Log in

A hybrid multi-population framework for dynamic environments combining online and offline learning

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Population based incremental learning algorithms and selection hyper-heuristics are highly adaptive methods which can handle different types of dynamism that may occur while a given problem is being solved. In this study, we present an approach based on a multi-population framework hybridizing these methods to solve dynamic environment problems. A key feature of the hybrid approach is the utilization of offline and online learning methods at successive stages. The performance of our approach along with the influence of different heuristic selection methods used within the selection hyper-heuristic is investigated over a range of dynamic environments produced by a well known benchmark generator as well as a real world problem, referred to as the Unit Commitment Problem. The empirical results show that the proposed approach using a particular hyper-heuristic outperforms some of the best known approaches in literature on the dynamic environment problems dealt with.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://www.asap.cs.nott.ac.uk/external/chesc2011/.

Abbreviations

EDA:

Estimation of distribution algorithms

HH-EDA2:

Hyper-heuristic based dual population EDA

PBIL:

Population based incremental learning

PBIL2:

Dual population PBIL

PBILr:

PBIL with restart

PBIL2r:

PBIL2 with restart

MPBILr:

Memory-based PBIL with restart

MPBIL2r:

Dual population memory-based PBIL with restart

Sentinel8:

Sentinel-based genetic algorithm with 8 sentinels

Sentinel16:

Sentinel-based genetic algorithm with 16 sentinels

DUF:

Decomposable unitation-based function

MW:

Megawatt

UCP:

Unit commitment problem

CL:

Cycle length

HF:

High frequency

HS:

High severity

LF:

Low frequency

LS:

Low severity

MF:

Medium frequency

MS:

Medium severity

VHS:

Very high severity

References

  • Baluja S (1994) Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA, Tech. rep

  • Barlow GJ, Smith SF (2009) Using memory models to improve adaptive efficiency in dynamic problems. In: IEEE symposium on computational intelligence in scheduling, CISCHED, pp 7–14

  • Bosman PAN (2005) Learning, anticipation and time-deception in evolutionary online dynamic optimization. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, ACM, GECCO ’05, pp 39–47

  • Branke J (1999) Memory enhanced evolutionary algorithms for changing optimization problems. In: IEEE congress on evolutionary computation CEC 99, vol 3, pp 1875–1882

  • Branke J (2002) Evolutionary optimization in dynamic environments. Kluwer, Norwell

  • Branke J, Kaussler T, Schmidt C, Schmeck H (2000) A multi-population approach to dynamic optimization problems. In: 4th International conference on adaptive computing in design and manufacture (ACDM 2000). Springer, Berlin, pp 299–308

  • Burke E, Kendall G (eds) (2005) Search methodologies: introductory tutorials in optimization and decision support techniques. Springer, Berlin

  • Burke EK, Gendreau M, Hyde MR, Kendall G, Ochoa G, Özcan E, Qu R (2013) Hyper-heuristics: A survey of the state of the art. J Oper Res Soc. doi:10.1057/jors.2013.71

  • Cao Y, Luo W, (2010) A novel updating strategy for associative memory scheme in cyclic dynamic environments. In: 3rd International workshop on advanced computational intelligence (IWACI), (2010), Suzhou, Jiangsu, pp 32–39

  • Chakhlevitch K, Cowling P (2008) Hyperheuristics: recent developments. Adaptive and multilevel metaheuristics. In: Cotta C, Sevaux M, Sirensen K (eds) Studies in computational intelligence, vol 136. Springer, Berlin, pp 3–29

  • Chen X, Ong YS, Lim MH, Tan KC (2011) A multi-facet survey on memetic computation. IEEE Trans Evolut Comput 15(5):591–607

    Article  Google Scholar 

  • Cheng H, Yang S (2010) Multi-population genetic algorithms with immigrants scheme for dynamic shortest path routing problems in mobile ad hoc networks. EvoApplications (1):562–571

  • Cowling PI, Kendall G, Soubeiga E (2000) A hyper-heuristic approach to scheduling a sales summit. In: 3rd International conference on practice and theory of automated timetabling III, PATAT 2000, Springer, Berlin, vol 2079

  • Crowston WB, Glover F, Thompson GL, Trawick JD (1963) Probabilistic and parametric learning combinations of local job shop scheduling rules. ONR Research memorandum, GSIA, Carnegie Mellon University, Pittsburgh, p 117

  • Cruz C, Gonzalez J, Pelta D (2011) Optimization in dynamic environments: a survey on problems, methods and measures. Soft Comput Fusion Found Methodol Appl 15:1427–1448

    Google Scholar 

  • Fernandes CM, Lima C, Rosa AC (2008a) Umdas for dynamic optimization problems. In: Proceedings of the 10th conference on genetic and evolutionary computation, ACM, GECCO ’08, pp 399–406

  • Fernandes CM, Lima C, Rosa AC (2008b) Umdas for dynamic optimization problems. In: Proceedings of the 10th annual conference on Genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’08, pp 399–406

  • Fisher H, Thompson GL (1963) Probabilistic learning combinations of local job-shop scheduling rules. In: Thompson GL (ed) Muth JF Industrial Scheduling. Prentice-Hall, Englewood Cliffs, pp 225–251

  • Ghosh A, Muehlenbein H (2004) Univariate marginal distribution algorithms for non-stationary optimization problems. Int J Know Based Intell Eng Syst 8(3):129–138

    Google Scholar 

  • Goncalves AR, Zuben FJV (2011) Online learning in estimation of distribution algorithms for dynamic environments. In : IEEE Congress on Evolutionary Computation, pp 62–69

  • Iacca G, Neri F, Mininno E, Ong YS, Lim MH (2012) Ockham’s razor in memetic computing: three stage optimal memetic exploration. Inf Sci 188:17–43

    Article  MathSciNet  Google Scholar 

  • Jin Y, Branke J (2005) Evolutionary optimization in uncertain environments-a survey. IEEE Trans Evolut Comp 9(3):303–317

    Article  Google Scholar 

  • Kiraz B, Topcuoglu HR (2010) Hyper-heuristic approaches for the dynamic generalized assignment problem. In: 10th International conference on intelligent systems design and Applications (ISDA), 2010, pp 1487–1492

  • Kiraz B, Etaner-Uyar AŞ, Özcan E (2011) An investigation of selection hyper-heuristics in dynamic environments. EvoApplications 2011. In: Lecture notes in computer science, vol 6624. Springer, Berlin, pp 314–323

  • Kiraz B, Etaner-Uyar AŞ, Özcan E (2013) Selection hyper-heuristics in dynamic environments. J Oper Res Soc. doi:10.1057/jors.2013.24

  • Kiraz B, Etaner-Uyar AŞ, Özcan E (2013) An ant-based selection hyper-heuristic for dynamic environments. EvoApplications 2013. In: Lecture notes in computer science, vol 7835. Springer, Berlin, pp 626–635

  • Kobliha M, Schwarz J, Očenášek J (2006) Bayesian optimization algorithms for dynamic problems. EvoWorkshops. In: Lecture notes in computer science, vol 3907. Springer, Berlin, pp 800–804

  • Larrañaga P, Lozano JA (eds) (2002) Estimation of distribution algorithms: a new tool for evolutionary computation. Kluwer, Boston

  • Lewis J, Hart E, Ritchie G (1998) A comparison of dominance mechanisms and simple mutation on nonstationary problems. In: Proceedings of parallel problem solving from nature, pp 139–148

  • Li X, Mabu S, Mainali M, Hirasawa K, (2011) Probabilistic model building genetic network programming using multiple probability vectors. In: TENCON, (2010) IEEE Region 10 Conference. IEEE Region Conference, Fukuoka, pp 1398–1403

  • Maturana J, Riff MC (2007) Solving the short-term electrical generation scheduling problem by an adaptive evolutionary approach. Eur J Oper Res 179:677–691

    Article  MATH  Google Scholar 

  • Morrison RW (2004) Designing evolutionary algorithms for dynamic environments. Springer, Berlin

  • Nareyek A (2004) Choosing search heuristics by non-stationary reinforcement learning. In: Metaheuristics. Kluwer Academic Publishers, Norwell, pp 523–544

  • Neri F, Cotta C (2012) Memetic algorithms and memeting computing optimization: a literature review. Swarm Evolut Comput 2:1–14

    Article  Google Scholar 

  • Neri F, Cotta C, Moscato P (eds) (2012) Handbook of Memetic Algorithms. In: Studies in computational intelligence. Springer, Berlin

  • Özcan E, Bilgin B, Korkmaz EE (2008) A comprehensive analysis of hyper-heuristics. Intell Data Anal 12:3–23

    Google Scholar 

  • Özcan E, Misir M, Ochoa G, Burke EK (2010) A reinforcement learning—great-deluge hyper-heuristic for examination timetabling. Int J Appl Metaheuristic Comput 1(1):39–59

    Article  Google Scholar 

  • Peng X, Gao X, Yang S (2011) Environment identification-based memory scheme for estimation of distribution algorithms in dynamic environments. Soft Comput 15:311–326

    Article  Google Scholar 

  • Ross P (2005) Hyper-heuristics. In: Burke EK, Kendall G (eds) Search methodologies: introductory tutorials in optimization and decision support techniques, vol 17. Springer, Berlin. pp 529–556

  • Saramourtsis A, Damousis J, Bakirtzis A, Dokopoulos P (1996) Genetic algorithm solution to the economic dispatch problem. IEE Proc C 141(4):377–382

    Google Scholar 

  • Shakya S, Oliveira F, Owusu G (2007) An application of eda and ga to dynamic pricing. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, New York, NY, USA, GECCO ’07, pp 585–592

  • Simões A, Costa E (2008a) Evolutionary algorithms for dynamic environments: Prediction using linear regression and markov chains. In: Proceedings of the 10th international conference on parallel problem solving from nature: PPSN X. Springer, Berlin, pp 306–315.

  • Simões A, Costa E (2008b) Evolutionary algorithms for dynamic environments: prediction using linear regression and markov chains. Tech. rep, Coimbra, Portugal

  • Simões A, Costa E (2009a) Improving prediction in evolutionary algorithms for dynamic environments. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’09

  • Simões A, Costa E (2009b) Prediction in evolutionary algorithms for dynamic environments using markov chains and nonlinear regression. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’09, pp 883–890

  • Uludağ G, Kiraz B, Etaner-Uyar AŞ, Özcan E (2012) A framework to hybridise PBIL and a hyper-heuristic for dynamic environments. In: PPSN 2012: 12th International conference on parallel problem solving from nature, vol 7492. Springer, Berlin, pp 358–367

  • Uludağ G, Kiraz B, Etaner-Uyar AŞ, Özcan E (2012b) Heuristic selection in a multi-phase hybrid approach for dynamic environments. In: 12th UK workshop on computational intelligence, Edinburgh, Scotland, UKCI12

  • Ursem RK (2000) Multinational GA optimization techniques in dynamic environments. In: Proceedings of the Genetic Evol. Comput. Conf, pp 19–26

  • Uyar Ş, Harmanci E (2005) A new population based adaptive domination change mechanism for diploid genetic algorithms in dynamic environments. Soft Comput 9(11):803–814

    Article  MATH  Google Scholar 

  • Uyar Ş, Türkay B (2008) Evolutionary algorithms for the unit commitment problem. Turk J Electr Eng 16(3):239–255

    Google Scholar 

  • Uyar Ş, Türkay B, Keleş A (2011) A novel differential evolution application to short-term electrical power generation scheduling. Electr Power Energy Syst 33:1236–1242

    Google Scholar 

  • Valenzuela J, Smith AE (2002) A seeded memetic algorithm for large unit commitment problems. J Heuristics 8:173–195

    Article  Google Scholar 

  • Wineberg M, Oppacher F (2000) Enhancing the GA’s ability to cope with dynamic environments. In: Whitley (ed) Genetic and evolutionary computation conference. Morgan Kaufmann, Burlington, pp 3–10

  • Wood AJ, Wollenberg BF (1996) Power generation, operation and control, 2nd edn. Wiley, New York

  • Wu Y, Wang Y, Liu X (2010a) Multi-population based univariate marginal distribution algorithm for dynamic optimization problems. J Intell Robotic Syst 59(2):127–144

    Google Scholar 

  • Wu Y, Wang Y, Liu X, Ye J (2010b) Multi-population and diffusion umda for dynamic multimodal problems. J Syst Eng Electron 21(5):777–783

    Google Scholar 

  • Xingguang P, Demin X, Fubin Z (2011) On the effect of environment-triggered population diversity compensation methods for memory enhanced umda. In: Proceedings of the 30th Chinese control conference, pp 5430–5435

  • Yang S (2004) Constructing dynamic test environments for genetic algorithms based on problem difficulty. In: In Proceedings of the 2004 congress on, evolutionary computation, pp 1262–1269

  • Yang S (2005a) Memory-enhanced univariate marginal distribution algorithms for dynamic optimization problems. In: Proceedings of the 2005 congress on, evolutionary computation pp 2560–2567

  • Yang S (2005b) Population-based incremental learning with memory scheme for changing environments. In: Proceedings of the 2005 conference on genetic and evolutionary computation, ACM, New York, NY, USA, GECCO ’05, pp 711–718

  • Yang S (2007) Explicit memory schemes for evolutionary algorithms in dynamic environments. In: Evolutionary computation in dynamic and uncertain environments, chap 1, pp 3–28

  • Yang S, Richter H (2009) Hyper-learning for population-based incremental learning in dynamic environments. In: Proc. 2009 Congr. Evol. Comput, pp 682–689

  • Yang S, Yao X (2005) Experimental study on population-based incremental learning algorithms for dynamic optimization problems. Soft Comput 9(11):815–834

    Article  MATH  Google Scholar 

  • Yang S, Yao X (2008) Population-based incremental learning with associative memory for dynamic environments. IEEE Trans Evolut Comp 12:542–561

    Article  Google Scholar 

  • Yang S, Ong YS, Jin Y (eds) (2007) Evolutionary computation in dynamic and uncertain environments. In: Studies in computational intelligence, vol 51. Springer, Berlin

  • Yuan B, Orlowska ME, Sadiq SW (2008) Extending a class of continuous estimation of distribution algorithms to dynamic problems. Optim Lett 2(3):433–443

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work is supported in part by the EPSRC, grant EP/F033214/1 (The LANCS Initiative Postdoctoral Training Scheme) and Berna Kiraz is supported by the TUBITAK 2211-National Scholarship Programme for PhD students.

Author information

Authors and Affiliations

  1. Institute of Science and Technology, Istanbul Technical University, Maslak, Istanbul, 34469, Turkey

    Gönül Uludağ & Berna Kiraz

  2. Department of Computer Engineering, Istanbul Technical University, Maslak, Istanbul, 34469, Turkey

    A. Şima Etaner-Uyar

  3. School of Computer Science, University of Nottingham, Nottingham, NG8 1BB, UK

    Ender Özcan

Authors
  1. Gönül Uludağ
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Berna Kiraz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Şima Etaner-Uyar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ender Özcan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ender Özcan.

Additional information

Communicated by G. Acampora.

A preliminary version of this study was presented at UKCI 2012: 12th Annual Workshop on Computational Intelligence.

Appendix: System data

Appendix: System data

Table 16 Generator data for System TR
Full size table
Table 17 Demand and reserve data for System TR
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Uludağ, G., Kiraz, B., Etaner-Uyar, A.Ş. et al. A hybrid multi-population framework for dynamic environments combining online and offline learning. Soft Comput 17, 2327–2348 (2013). https://doi.org/10.1007/s00500-013-1094-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-013-1094-7

Keywords

Search

Navigation