Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization

https://doi.org/10.1016/j.ejor.2005.01.022Get rights and content

Abstract

A numerical solution to a 30-dimensional water reservoir network optimization problem, based on stochastic dynamic programming, is presented. In such problems the amount of water to be released from each reservoir is chosen to minimize a nonlinear cost (or maximize benefit) function while satisfying proper constraints. Experimental results show how dimensionality issues, given by the large number of basins and realistic modeling of the stochastic inflows, can be mitigated by employing neural approximators for the value functions, and efficient discretizations of the state space, such as orthogonal arrays, Latin hypercube designs and low-discrepancy sequences.

Introduction

Optimal operation of water reservoir networks has been studied extensively in literature (e.g., [1], [12], [13], [21], [34]; Yakowitz [37] provides an excellent survey). Reservoir networks systems can be represented by graphs in which the nodes correspond to water basins and the links are characterized by interbasin transfers. In typical models, the inputs to the nodes are water released from upstream reservoirs and stochastic inflows from external sources (like rivers and rain), while the outputs correspond to the amount of water to be released during a given time period (e.g., a month).

The dynamics for the single basin can be modeled by a state equation where the amount of water at the beginning of period t + 1 reflects the flow balance between the water that enters (upstream releases and stochastic inflows) and the water that is released during period t. The amount of water to be released during a given time period from each reservoir is chosen to minimize some possibly nonlinear cost (or maximize benefit) function related to the releases (e.g., power generation), while satisfying proper constraints, e.g., maximum pumpage capacities or target water level in each basin at the beginning of each period t. Thus, optimal management of the reservoir network can be formulated as an optimization problem, in which the aim is to determine the quantity of water releases that minimize a total cost over a given horizon of T time periods (e.g., a year). In the case of large-scale reservoir networks with several basins, nonlinearities and the presence of stochastic variables, the corresponding optimization problem becomes very complex.

This is particularly evident when we want to model the stochastic inflows accurately. In a realistic representation of inflow dynamics, the amount of (random) water flowing into the basins during a given period t depends on the amounts of past periods. An example of realistic modeling, widely and successfully employed in the water resources literature [13], [30], is to consider autoregressive linear models of a given order k. To construct this model, the inflows of the past k time periods must be included in the state vector, which consequently can become very large.

Stochastic dynamic programming (SDP) [3], [4], [28] is the most commonly used solution technique in the reservoir networks management literature [1], [12], [13], [20], [27], [37]. SDP is based on the definition, at each stage t, of a value function which quantifies the cost from that stage through the end of the time horizon. In this way, it is possible to transform the optimization problem into the recursive solution of a sequence of simpler optimization subproblems. It is well known that an exact solution to the SDP equations can be obtained only when system dynamics are linear and the cost (benefit) function is quadratic. In the general case we must seek approximate solutions, which are based on a state space discretization and an approximation of the value functions over the continuous space. Although for the deterministic case there exist efficient versions of dynamic programming, such as Differential DP [19], classical approaches for the stochastic case suffer from the curse of dimensionality phenomenon, which is an exponential growth of the computational and memory requirements as the dimension of the state vector increases. This is why only reservoir networks of limited size are considered in the literature, in the absence of restrictive hypotheses on the model. For example, in a recent paper [27], the largest example involved seven state variables. Therefore, for large-scale reservoir networks with accurate inflow dynamics, a very efficient version of SDP is still needed.

In this work we present approximate solutions to a 30-dimensional problem. The state space dimension arises from a test network with 10 reservoirs and inflows modeled by an autoregressive system of order 2, where the cost and the constraints are nonlinear. To the best of the authors’ knowledge, this is the largest dimensional problem ever addressed in the reservoirs management literature by SDP techniques, at least without introducing restrictive hypotheses on the cost and/or the model.

For solving such a high-dimensional problem we utilize an approach based on efficient discretization of the state space and approximation of the value functions over the continuous state space by means of a flexible feedforward neural network. Other methods that employ neural networks for reservoir control problems can be found in literature. For instance, in [31], a 10-dimensional model where the cost function represents energy deficit in power generation is solved by an algorithm based on a two-phase neural network.

The approach presented here is based on the high-dimensional continuous-state SDP work of Chen [7], [8], [9] and Baglietto et al. [38]. The most commonly employed discretization in the literature consists of a uniform grid of points over the state space, which is not efficient because it is subject to exponential growth. Referring to the area of statistical design of experiments, we consider more efficient discretizations. Specifically, orthogonal arrays (OA) [8], OA-based Latin hypercubes (OA-LH) [33], and number-theoretic methods based on Sobol′ [32] and Niederreiter–Xing [25], [26] low-discrepancy sequences. For what concerns low-discrepancy sequences in particular, it has been recently proven [6] that their use in the context of function learning leads to an almost linear sample complexity in the estimation of the best network inside the chosen family, which is in general one of the main sources of curse of dimensionality when value functions have to be approximated in SDP problems.

The paper is organized as follows. Section 2 contains a description of the 30-dimensional test reservoir network model. In Section 3 the method based on SDP is introduced and discussed. Section 4 is devoted to the actual solution of the test problem, where the relevant parameters and results are presented. Section 5 contains some concluding remarks.

Section snippets

The model

The water reservoir network we consider consists of 10 basins, each one affected by stochastic inflows and controlled by means of water releases. The configuration of the network is depicted in Fig. 1. The details concerning the model, defined in the following subsections, reflect typical situations that can be found in the reservoirs management literature.

SDP solution to the optimization problem

The SDP approach for the solution of a finite-horizon optimization problem is based on the definition of a value function for a given time period, which corresponds to the optimal cost to operate the system from the current time period through the end of the time horizon. This value function is a function of the state variables, i.e., the optimal cost to operate the system depends on the states. Formally, the value function can be written recursively asFt(xt)=minrtEϵt[ht(xt,rt,ϵt)+Ft+1(xt+1)]s.t

Feedforward neural networks

As known, the basic architecture of feedforward neural networks is an interconnection of neural nodes that are organized in layers. The input layer consists of n nodes, one for each input variable, and the output layer consists of one node for the output variable. In between there are hidden layers which induce flexibility into the modeling. Activation functions define transformations between layers, the simplest one being a linear function. A comprehensive description of various forms may be

Conclusions

A computationally-tractable SDP method for the optimal management of large-scale water reservoir networks was presented. The use of innovative discretization schemes, such as OAs and (t, m, s)-nets, allows to deal effectively with the curse of dimensionality problems, arising from the number of basins and realistic modeling of the random inflows. Four different design types have been tested and compared in a 30-dimensional test problem. The employment of such discretization schemes has allowed

Acknowledgements

This work was partially supported by NSF Grants #INT 0098009 and #DMI 0100123 and a Technology for Sustainable Environment (TSE) grant under the US Environmental Protection Agency’s Science to Achieve Results (STAR) program (Contract #R-82820701-0).

References (38)

  • V.C.P. Chen

    Measuring the goodness of orthogonal array discretizations for stochastic programming and stochastic dynamic programming

    SIAM Journal of Optimization

    (2001)
  • V.C.P. Chen et al.

    Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming

    Operations Research

    (1999)
  • K.-T. Fang et al.

    Number-theoretic Methods in Statistics

    (1994)
  • E. Foufoula-Georgiou et al.

    Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems

    Water Resources Research

    (1988)
  • S. Gal

    Optimal management of a multireservoir water supply system

    Water Resources Research

    (1979)
  • A.P. Georgakakos

    Extended linear quadratic Gaussian control for the real time operation of reservoir systems

  • M.T. Hagan et al.

    Training feedforward networks with the marquardt algorithm

    IEEE Transactions on Neural Networks

    (1994)
  • S. Haykin

    Neural Networks: A Comprehensive Foundation

    (1999)
  • W. Hoeffding

    Probability inequalities for sum of bounded random variables

    Journal of the American Statistical Association

    (1963)
  • Cited by (100)

    • Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization

      2020, Computers and Operations Research
      Citation Excerpt :

      In observational data studies, there is no control over data collection; however, in computer experiments, data collection is controlled with the aim to improve the ability of the statistical modeling task to uncover a high-quality representation with minimal data. Thus far, two other state space sampling methods using design of experiments, namely, Latin hypercube and low-discrepancy sequences, have also been studied in the literature (e.g., Cervellera et al., 2006). Note that besides the DACE based sampling techniques, other sampling methods have also shown outstanding performance in the literature, namely importance sampling (Papavasiliou and Shmuel, 2013), simulation-based sampling (Tilmant et al., 2008), scenario sampling (Zéphyr et al., 2017), and simplicial-based sampling (Zéphyr et al., 2015).

    • The economics of aquifer protection plans under climate water stress: New insights from hydroeconomic modeling

      2019, Journal of Hydrology
      Citation Excerpt :

      Despite the advantages of using hydroeconomic optimization to more efficiently manage effects of climate and policy futures on the dynamics of water use patterns in the Rio Grande Basin, the model has limitations we plan to address in future work. Dynamic stochastic optimization (Cervellera et al., 2006) would provide a more comprehensive understanding of the effects of climate stress and of policies to promote resilience, as well as uncertainties related to the economic and hydrological parameters that affect outcomes, though the paper handles some of this. While the data used for this paper are recent and more comprehensive than of previous published studies we were able to find, data limitations with respect to variability over time and space will influence the results.

    • A stochastic program with time series and affine decision rules for the reservoir management problem

      2018, European Journal of Operational Research
      Citation Excerpt :

      This is a huge improvement over stochastic dynamic programming (SDP) methods, which have historically been the most popular techniques used for reservoir management both in academia and in practice (see Castelletti, Pianosi, and Soncini-Sessa (2008); Labadie (2004); Turgeon (2005); Yeh (1985) and references therein). Although these methods can deal with more realistic non-convex optimization problems and provide excellent closed-loop policies, they can usually only consider serial correlation through autoregressive models of small order since higher order models require increasing the state-space, which quickly leads to numerical intractability; particularly for multi-reservoir operations (Cervellera, Chen, & Wen, 2006; Tilmant & Kelman, 2007; Turgeon, 2007). Considering heteroscedasticity further increases the state dimension and the resulting computational burden for SDP.

    View all citing articles on Scopus
    View full text