Evolutionary algorithm enhancement for model predictive control and real-time decision support

https://doi.org/10.1016/j.envsoft.2015.03.005Get rights and content

Abstract

Effective decision support and model predictive control of real-time environmental systems require that evolutionary algorithms operate more efficiently. A suite of model predictive control (MPC) genetic algorithms are developed and tested offline to explore their value for reducing combined sewer overflow (CSO) volumes during real-time use in a deep-tunnel sewer system. MPC approaches include the micro-GA, the probability-based compact GA, and domain-specific GA methods that reduce the number of decision variable values analyzed within the sewer hydraulic model, thus reducing algorithm search space. Minimum fitness and constraint values achieved by all GA approaches, as well as computational times required to reach the minimum values, are compared to large population sizes with long convergence times. Optimization results for a subset of the Chicago combined sewer system indicate that genetic algorithm variations with a coarse decision variable representation, eventually transitioning to the entire range of decision variable values, are best suited to address the CSO control problem. Although diversity-enhancing micro-GAs evaluate a larger search space and exhibit shorter convergence times, these representations do not reach minimum fitness and constraint values. The domain-specific GAs prove to be the most efficient for this case study. Further MPC algorithm developments are suggested to continue advancing computational performance of this important class of problems with dynamic strategies that evolve as the external constraint conditions change.

Introduction

With increasing availability of data and information in near-real time, computationally efficient model predictive control (MPC) algorithms are needed to improve real-time management of large-scale, dynamic environmental systems (Maier et al., 2014). MPC (or receding horizon control) involves forecasting the future state of the system for an operational decision window and using a time-varying objective function to identify optimal solutions for the next decision window (Jin and Branke, 2005). This work focuses on a suite of genetic algorithm (GA) MPC approaches and tests their performance in optimizing hydraulics of combined sewer systems, which change rapidly due to shifts in rainfall and their forecasts. In pursuing these objectives, this work also presents new modifications to genetic algorithms that reduce the search space of the problem in order to improve computational efficiency.

Genetic algorithms (Holland, 1975, Goldberg, 1989) are search techniques that identify optimal or near-optimal solutions using operations analogous to natural selection with a population of chromosomes; each chromosome represents a possible solution. Genetic algorithm evolution is based on assembling building blocks, or components, of good solutions (Goldberg, 2002). A GA is implicitly parallel (Goldberg, 1989) because within a population, the GA can process many of these building blocks at the same time. Several different blocks can remain in solution that each represent a subset of a good solution; over time these building blocks are combined into the optimal solution. As a result of utilizing building blocks, GAs prove more efficient than enumeration algorithms (Goldberg, 2002) and avoid the curse of dimensionality encountered in dynamic programming (Michalewicz et al., 1992). Genetic algorithms undergo probability-based selection; the likelihood of an individual to undergo reproduction is a function of its fitness (Goldberg, 1989, Cai et al., 2001). Goldberg (1989) asserts that due to inclusion of crossover, the GA does better than hill-climbing, or local search through incremental changes to the solution. Genetic algorithms are beneficial for solving environmental problems due to their ability to solve nonlinear and discontinuous optimization problems for which gradient-based methods can find only locally optimal solutions (Celeste et al., 2004, Nicklow et al., 2010). GAs also have extensive theory to support effective parameterization (Reed et al., 2000, Minsker, 2005), are easy to connect with non-linear physics-based models, and are widely used. Note that other evolutionary algorithms and non-population based methods could also be used to account for non-linearity within an optimization framework.

GAs have been implemented within MPC frameworks for several environmental applications. Muleta and Nicklow (2005) coupled a genetic algorithm and MPC within the United States Department of Agriculture's Soil and Water Assessment Tool (SWAT) model and an artificial neural network (ANN) to determine optimal crop types for a 3-year planning horizon. Dhar and Datta (2007) minimized the deviation between target and actual reservoir levels in order to control downstream water quality. Celeste et al. (2004) also used GA MPC to optimize reservoir operation releases, while Rauch and Harremoes (1999) applied GA MPC to maximize the mean dissolved oxygen concentration below an urban wastewater system. Additional work has applied GA MPC to a wide range of studies outside the environmental field: operation of an autonomous underwater vehicle (Naeem et al., 2005), a laboratory fermenter (Onnen et al., 1997), and real-time traffic control signals (Lee et al., 2005, Memon and Bullen, 1996). Hu and Chen (2005) apply GA MPC for aircraft arrival sequencing and scheduling.

During MPC, a strategy for the decision (in this case also the forecast) horizon is developed using the simulation model during the first time interval. The first interval of that optimized strategy is implemented while a new forecast is obtained and the next strategy is found. Although MPC approaches offer significant advantage for enabling near-real-time control, the computational demands of applying heuristic algorithms such as GAs using an MPC approach can be daunting, particularly for complex non-linear problems such as combined sewer overflow (CSO) control. To address this problem, a suite of model predictive control genetic algorithms are developed in this work and tested offline to explore their value for reducing computational time to minimize CSO volumes in near-real time. The MPC algorithms assign sluice gate positions and pumping rates that minimize CSO flows and limit high flows that lead to hydraulic instabilities during spatially and temporally variable storm events using a numerical hydraulic model.

This paper explores how MPC GAs computational performance can be improved, as recommended by Maier et al. (2014). Performance is improved by limiting fitness evaluations using the following algorithmic approaches: the micro-GA (Krishnakumar, 1989, Pico and Wainwright, 1994, Coello and Pulido, 2001), the probability-based compact GA (Mininno et al., 2008), memory enhancements specific to MPC (Onnen et al., 1997), and domain-specific methods that reduce the number of decision variables values. The latter approaches were inspired by multiscale GAs developed by Babbar and Minsker (2006) and Sinha and Minsker (2007) and noted by Maier et al. (2014) as important to reducing the search space. All new GA approaches are compared to simple genetic algorithms with larger population sizes determined by GA theory (Reed et al., 2000, Minsker, 2005).

The algorithms are tested on a hydraulic model that simulates flow in a portion of the Chicago combined sewer system and deep tunnel along the northern portion of the Chicago River. The fitness, or objective function value, of a chromosome (set of decision variable values, or real-coded genes) is computed using a hydraulic simulation model (Storage Routing Model, SRM; Zimmer et al., 2013).

Section snippets

Case study

This section presents the location of the case study in Chicago where the MPC GA algorithms were tested and information on the storm event used for comparing the algorithms.

Methodology

The methodology consists of three aspects, described separately below: the MPC formulation, the real-coded genetic algorithm implementation, and the enhancements made to the genetic algorithm to improve computational performance. These sections were presented after the case study to facilitate description of gene descriptions and equations applied.

Genetic algorithm implementation

Because the population size used in this algorithm is fixed during each run (and does not change based on environmental parameters), a high mutation rate is necessary to expand the breadth of the search, at least for early generations (Tate and Smith, 1993, Wright, 1991). Wright (1991) supports higher mutation rates and uses a range of 0.04–0.3; a mutation rate of 0.05 for this work was derived from these values and further exploration during the initial stages of model development.

Real-coded

Algorithm modifications to improve performance

Given that large-scale sewer systems can require significant computational effort to optimize, particularly in near-real time, this section presents modifications to the real-coded genetic algorithm for enhanced efficiency. Parameters used within all variations of the real-coded GA are discussed briefly, and the performance of the GA modifications is compared in the results section of this paper.

The crossover and mutation operators discussed above are implemented within all variations of the GA

Results and discussion

The various GA modifications are compared to identify the best performing approach for this case study. To address random variability that can occur with GA runs, ten different random number seeds are applied to each method. Mean and 75th and 25th percentile values are reported to summarize results across all seeds for each method, as shown in Fig. 12. The combination of lines and points (crosses) for each GA modification is centered at the mean values for the 10 GA runs and the lines through

Conclusions

The methods tested in this paper yield effective and computationally efficient MPC GAs with which to analyze time-varying systems with moving decision windows. A range of adaptations to the simple GA are presented, from those that maintain the most diversity (such as the micro-GA), but do not find the optimal solution to the problem despite a short computational time, to those that work primarily as hill-climbing and local search algorithms (the cGA) and run with short durations but find

Acknowledgements

Funding for the first author was provided through an IBM Ph.D. fellowship. The Metropolitan Water Reclamation District of Greater Chicago (MWRDGC) provided partial funding and the CSO data from which the case study is derived.

References (51)

  • S. Baluja

    Population-based Incremental Learning: a Method for Integrating Genetic Search Based Function Optimization and Competitive Learning

    (1994)
  • A.B. Celeste et al.

    Genetic algorithms for real-time operation of multipurpose water resource systems

    J. Hydroinformatics

    (2004)
  • F. Chang et al.

    Real-coded genetic algorithm for rule-based flood control reservoir management

    Water Resour. Manag.

    (1998)
  • C.A. Coello et al.

    A micro-genetic algorithm for multiobjective optimization

  • A. Dhar et al.

    Optimal operation of reservoirs for downstream water quality control using linked simulation optimization

    J. Hydrol. Process

    (2007)
  • S. Duchense et al.

    Mathematical Modeling of Sewers Under Surcharge for Real Time Control of Combined Sewer Overflows

    (2001)
  • Engineering Toolbox, Hydraulic Pump Power. www.engineeringtoolbox.com/pumps-power-d_505.html. (accessed...
  • L. Eshelman et al.

    Real-coded genetic algorithms and interval-Schemata

  • S. Glovick et al.

    The formation and control of surges and geysers in tunnel systems

  • D.E. Goldberg

    Genetic Algorithms in Search, Optimization, and Machine Learning

    (1989)
  • D.E. Goldberg

    Real-coded Genetic Algorithms, Virtual Alphabets, and Blocking

    (1990)
  • D.E. Goldberg

    The Design of Innovation: Lessons from and for Competent Genetic Algorithms

    (2002)
  • Q. Guo et al.

    Surging in urban storm drainage systems

    J. Hydraul. Eng.

    (1990)
  • G.R. Harik et al.

    The compact genetic algorithm

    IEEE Trans. Evol. Comput.

    (Nov.1999)
  • F. Herrera et al.

    Tackling real-coded genetic algorithms: operators and tools for behavioral analysis

    Artif. Intell. Rev.

    (1998)
  • Cited by (32)

    • Real-time control of combined sewer systems: Risks associated with uncertainties

      2023, Journal of Hydrology
      Citation Excerpt :

      This mimics the observed, i.e. actual operational capacity available during the event. The precipitation forecast parsed to the internal MPC model is either the observed precipitation representing a perfect forecast (e.g. Zimmer et al. 2015) or the actual, i.e. real historical forecasts (Löwe et al. 2016). To understand the influence of the previously highlighted uncertainties in the system, a generic methodology was set up.

    • Impact of sewer overflow on public health: A comprehensive scientometric analysis and systematic review

      2022, Environmental Research
      Citation Excerpt :

      Disadvantages of Monte Carlo include output accuracy depends on utilization of reasonable/fair assumptions, tendency to underestimate risk events, computational requirements, time-consuming and susceptible to overfitting (Dilks et al., 1992; Han et al., 2007; Thorndahl et al., 2008). Genetic algorithm (GA) is an efficient algorithm/tool inspired by nature for real-time optimization of the sewer network system for effective decision making to control SO (Bonamente et al., 2020; Zimmer et al., 2015). GA has advantages of flexibility, prompt adaptation to changing conditions and reliability and limited CPU requirements (Bonamente et al., 2020).

    • Dynamic control of urban sewer systems to reduce combined sewer overflows and their adverse impacts

      2019, Journal of Hydrology
      Citation Excerpt :

      Some other researchers have introduced model predictive control (MPC) approaches to minimize CSOs (Joseph-Duran et al., 2015; Zhao et al., 2017; Snodgrass et al., 2018). Zimmer et al. (2015) presented an MPC model to reduce the CSOs for a deep-tunnel sewer system. They have further extended their work (Zimmer et al., 2018) to explore the efficiency and effectiveness of different MPC approaches.

    • Leveraging open source software and parallel computing for model predictive control of urban drainage systems using EPA-SWMM5

      2019, Environmental Modelling and Software
      Citation Excerpt :

      Because we used EPA-SWMM5 as a black-box process model, a metaheuristic was used in place of a true optimization procedure to find an effective control policy at each time step in the MPC run. We chose an evolutionary algorithm (EA) for the metaheuristic since it has been shown to be successful in other urban drainage control applications (Zimmer et al., 2015, 2018) and it's inherent propensity for parallelization Maier et al. (2014). An EA begins with an initial population of individuals where, in our case, each individual is a control policy.

    View all citing articles on Scopus

    Thematic Issue on the Evolutionary Algorithms.

    View full text