Abstract
Two methods of approximate solution are developed for T-stage stochastic optimal control (SOC) problems, aimed at obtaining finite-horizon management policies for water resource systems. The presence of uncertainties, such as river and rain inflows, is considered. Both approaches are based on the use of families of nonlinear functions, called “one-hidden-layer networks” (OHL networks), made up of linear combinations of simple basis functions containing parameters to be optimized. The first method exploits OHL networks to obtain an accurate approximation of the cost-to-go functions in the dynamic programming procedure for SOC problems. The approximation capabilities of OHL networks are combined with the properties of deterministic sampling techniques aimed at obtaining uniform samplings of high-dimensional domains. In the second method, admissible solutions to SOC problems are constrained to take on the form of OHL networks, whose parameters are determined in such a way to minimize the cost functional associated with SOC problems. Exploiting these tools, the two methods are able to cope with the so-called “curse of dimensionality,” which strongly limits the applicability of existing techniques to high-dimensional water resources management in the presence of uncertainties. The theoretical bases of the two approaches are investigated. Simulation results show that the proposed methods are effective for water resource systems of high dimension.
Similar content being viewed by others
References
Garstka, S.J., Wets, R.J.B.: On decision rules in stochastic programming. Math. Program. 7, 117–173 (1974)
Rockafellar, R.T., Wets, R.J.-B.: Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16, 119–147 (1991)
Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957)
Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996)
Larson, R.: State Increment Dynamic Programming. Elsevier, New York (1968)
Jacobson, D., Mayne, D.: Differential Dynamic Programming. Academic Press, New York (1970)
Turgeon, A.: A decomposition method for the long-term scheduling of reservoirs in series. Water Resour. Res. 17, 1565–1570 (1981)
Archibald, T.W., McKinnon, K.I.M., Thomas, L.C.: An aggregate stochastic dynamic programming model of multireservoir systems. Water Resour. Res. 33, 333–340 (1997)
Bellman, R., Kalaba, R., Kotkin, B.: Polynomial approximation—A new computational technique in dynamic programming. Math. Comput. 17, 155–161 (1963)
Foufoula-Georgiou, E., Kitanidis, P.K.: Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems. Water Resour. Res. 24, 1345–1359 (1988)
Johnson, S.A., Stedinger, J.R., Shoemaker, C., Li, Y., Tejada-Guibert, J.A.: Numerical solution of continuous-state dynamic programs using linear and spline interpolation. Oper. Res. 41, 484–500 (1993)
Philbrick, Jr., C.R., Kitanidis, P.K.: Improved dynamic programming methods for optimal control of lumped-parameter stochastic systems. Oper. Res. 49, 398–412 (1999)
Chen, V.C.P., Ruppert, D., Shoemaker, C.A.: Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming. Oper. Res. 47, 38–53 (1999)
Longstaff, F.A., Schwartz, E.S.: Valuing American options by simulation: A simple least squares approach. Rev. Financ. Stud. 14, 113–147 (2001)
Carriere, J.F.: Valuation of the early-exercise price for derivative securities using simulations and splines. Insur. Math. Econ. 19, 19–30 (1996)
Brodie, P., Glasserman, M.: A stochastic mesh method for pricing high dimensional American options, J. Comput. Finance 7, 2004
Zoppoli, R., Parisini, T.: Learning techniques and neural networks for the solution of n-stage nonlinear nonquadratic optimal control problems. In: Isidori, A., Tarn, T.J. (eds.) Systems, Models and Feedback: Theory and Applications, pp. 193–210. Birkhäuser, Boston (1992)
Alessandri, A., Parisini, T., Sanguineti, M., Zoppoli, R.: Neural strategies for nonlinear optimal filtering. In: Proc. IEEE Int. Conf. Syst. Eng., Kobe, Japan, 1992, pp. 44–49
Zoppoli, R., Parisini, T., Sanguineti, M.: Approximating networks and extended Ritz method for the solution of functional optimization problems. J. Optim. Theory Appl. 112, 403–439 (2002)
Kůrková, V., Sanguineti, M.: Comparison of worst-case errors in linear and neural network approximation. IEEE Trans. Inf. Theory 48, 264–275 (2002)
Kůrková, V., Sanguineti, M.: Error estimates for approximate optimization by the extended Ritz method. SIAM J. Optim. 15, 261–287 (2005)
Cervellera, C., Muselli, M.: Deterministic design for neural–network learning: An approach based on discrepancy. IEEE Trans. Neural Netw. 15, 533–544 (2004)
Yakowitz, S.: Dynamic programming applications in water resources. Water Resour. Res. 18, 673–696 (1982)
Cervellera, C., Chen, V.C., Wen, A.: Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization. Eur. J. Oper. Res. 171(3), 1139–1151 (2006)
Gal, S.: Optimal management of a multireservoir water supply system. Water Resour. Res. 15, 737–749 (1979)
Salas, J.D., Tabios III, G.Q., Bartolini, P.: Approaches to multivariate modeling of water resources time series. Water Resour. Bull. 21, 683–708 (1985)
Grippo, L.: Convergent on-line algorithms for supervised learning in neural networks. IEEE Trans. Neural Netw. 11, 1284–1299 (2000)
Alessandri, A., Sanguineti, M.: Optimization-based learning with bounded error for feedforward neural networks. IEEE Trans. Neural Netw. 13, 261–273 (2002)
Alessandri, A., Cuneo, M., Pagnan, S., Sanguineti, M.: A recursive algorithm for nonlinear least-squares problems. Comput. Optim. Appl. 38, 195–216 (2007)
Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39, 930–945 (1993)
Barron, A.R.: Approximation and estimation bounds for artificial neural networks. Mach. Learn. 14, 115–133 (1994)
Kushner, H.J., Yin, G.G.: Stochastic Approximation Algorithms and Applications. Springer, New York (1997)
Leshno, M., Ya, V., Pinkus, A., Schocken, S.: Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw. 6, 861–867 (1993)
Park, J., Sandberg, I.W.: Universal approximation using radial-basis-function networks. Neural Comput. 3, 246–257 (1991)
Rudin, W.: Principles of Mathematical Analysis, 3rd edn. McGraw-Hill, New York (1976)
Thenie, J., Vial, J.P.: Step decision rules for multistage stochastic programming: A heuristic approach. Automatica 44, 1569–1584 (2008)
Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Prentice-Hall, Englewood Cliffs (1963)
Giulini, S., Sanguineti, M.: Approximation schemes for functional optimization problems J. Optim. Theory Appl. doi:10.1007/s10957-008-9471-6
Parisini, T., Zoppoli, R.: Neural networks for feedback feedforward nonlinear control systems. IEEE Trans. Neural Netw. 5, 436–449 (1994)
Zoppoli, R., Parisini, T.: Neural approximations for multistage optimal control of nonlinear stochastic systems. IEEE Trans. Autom. Control 41, 889–895 (1996)
Baglietto, M., Parisini, T., Zoppoli, R.: Distributed–information neural control: the case of dynamic routing in traffic networks. IEEE Trans. Neural Netw. 12, 485–502 (2001)
Baglietto, M., Sanguineti, M., Zoppoli, R.: Facing the curse of dimensionality by the extended Ritz method in stochastic functional optimization: dynamic routing in traffic networks. In: Di Pillo, G., Murli, A. (eds.) High Performance Algorithms and Software for Nonlinear Optimization, pp. 23–56. Kluwer Academic, Dordrecht (2003)
Alessandri, A., Sanguineti, M.: Optimization of approximating networks for optimal fault diagnosis. Optim. Methods Softw. 20, 241–266 (2005)
Wang, L.Y., Yin, G.G.: Persistent identification of systems with unmodeled dynamics and exogenous disturbances. IEEE Trans. Autom. Control 45, 1246–1256 (2000)
Niyogi, P., Girosi, F.: On the relationship between generalization error, hypothesis complexity, and sample complexity for radial basis functions. Neural Comput. 8, 819–842 (1996)
Nussbaum, M.: On nonparametric estimation of a regression function that is smooth in a domain on ℝk. Theory Probab. Appl. 31, 118–125 (1986)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–269 (1995)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia (1992)
Cervellera, C., Muselli, M.: Efficient sampling in approximate dynamic programming algorithms. Comput. Optim. Appl. 38, 417–443 (2007)
Fang, K.-T., Wang, Y.: Number-Theoretic Methods in Statistics. Chapman & Hall, London (1994)
Sobol’, I.M.: The distribution of points in a cube and the approximate evaluation of integrals. Zh. Vychisl. Mat. Mat. Fiz. 7, 784–802 (1967)
Hagan, M.T., Menhaj, M.: Training feedforward networks with the Marquardt algorithm. IEEE Trans. Neural Netw. 5, 989–993 (1994)
Kůrková, V., Sanguineti, M.: Learning with generalization capability by kernel methods of bounded complexity. J. Complex. 21, 350–367 (2005)
Gnecco, G., Sanguineti, M.: Accuracy of suboptimal solutions to kernel principal component analysis, Comput. Optim. Appl. doi:10.1007/s10589-007-9108-y
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Baglietto, M., Cervellera, C., Sanguineti, M. et al. Management of water resource systems in the presence of uncertainties by nonlinear approximation techniques and deterministic sampling. Comput Optim Appl 47, 349–376 (2010). https://doi.org/10.1007/s10589-008-9221-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-008-9221-6