Abstract
This paper is concerned with grid-level battery storage operations, taking battery aging into consideration. Battery operations under price uncertainty are modeled as a Markov Decision Process with expected cumulated discounted rewards. The structure of the optimal policy is studied. An algorithm that takes advantage of the problem structure and works directly on the continuous state space is developed to maximize the objective over the life of the battery. The algorithm determines an optimal policy by solving a sequence of quasiconvex problems indexed by a battery-life state. Computational results are presented to compare the proposed approach to a standard dynamic programming method, and to evaluate the impact of refinements in the battery model. Error bounds for the proposed algorithm are established to demonstrate its accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akhil, A., Huff, G., Currier, A., Kaun, B., Rastler, D., Chen, S., Cotter, A., Bradshaw, D., & Gauntlett, W. (2015). DOE/EPRI electricity storage handbook in collaboration with NRECA. Tech. Rep. SAND2015-1002, Sandia National Laboratories
Bertsekas, D. (2005). Dynamic programming and optimal control (3rd ed.). Belmont, MA: Athena Scientific.
Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
Dontchev, A., & Rockafellar, R. (2009). Implicit functions and solution mappings. New York, NY: Springer.
Dunn, B., Kamath, H., & Tarascon, J. M. (2011). Electrical energy storage for the grid: A battery of choices. Science, 334(6058), 928–935.
Greenberg, H., & Pierskalla, W. (1973). Quasi-conjugate functions and surrogate duality. Cahiers du Centre d’Etudes de Recherche Opérationnelle, 15, 437–448.
Heyman, D., & Sobel, M. (2003). Stochastic models in operations research, vol II: Stochastic optimization. Mineola, NY: Dover.
Hu, Y., & Defourny, B. (2014). Near-optimality bounds for greedy periodic policies with application to grid-level storage. In IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL-2014), pp. 1–8
Kallenberg, L. (2002). Finite state and action MDPs. In E. Feinberg & A. Schwartz (Eds.), Handbook of Markov decision processes (pp. 21–87). Boston: Kluwer.
Kiwiel, K. (2001). Convergence and efficiency of subgradient methods for quasiconvex minimization. Mathematical Programming, 90(1), 1–25.
Koller, M., Borsche, T., Ulbig, A., & Andersson, G. (2013). Defining a degradation cost function for optimal control of a battery energy storage system. In 2013 IEEE PowerTech, pp. 1–6
Lifshitz, D., & Weiss, G. (2015). Optimal energy management for grid-connected storage systems. Optimal Control Applications and Methods, 36(4), 447–462.
Löhndorf, N., & Minner, S. (2010). Optimal day-ahead trading and storage of renewable energies—An approximate dynamic programming approach. Energy Systems, 1(1), 61–77.
Magnus, J., & Neudecker, H. (1999). Matrix differential calculus (revised ed.). Chichester: Wiley.
Moazeni, S., Powell, W., & Hajimiragha, A. (2015). Mean-conditional value-at-risk optimal energy storage operation in the presence of transaction costs. IEEE Transactions on Power Systems, 30(3), 1222–1232.
Mount, T., Ning, Y., & Cai, X. (2006). Predicting price spikes in electricity markets using a regime-switching model with time-varying parameters. Energy Economics, 28(1), 62–80.
Nocedal, J., & Wright, S. (2006). Numerical optimization (2nd ed.). New York: Springer.
Petrik, M., & Wu, X. (2015). Optimal threshold control for energy arbitrage with degradable battery storage. In Uncertainty in Artificial Intelligence, Proc. 31st Conf. (pp. 692–701). Corvallis: AUAI Press.
Plemmons, R. (1977). M-matrix characterizations. I—Nonsingular M-matrices. Linear Algebra and its Applications, 18(2), 175–188.
Puterman, M. (1994). Markov decision processes: Discrete stochastic dynamic programming. Hoboken, NJ: Wiley.
Puterman, M., & Brumelle, S. (1979). On the convergence of policy iteration in stationary dynamic programming. Mathematics of Operations Research, 4(1), 60–69.
Qin, J., Sevlian, R., Varodayan, D., & Rajagopal, R. (2012). Optimal electric energy storage operation. In 2012 IEEE Power and Energy Society General Meeting (pp. 1–6) IEEE
Ross, S. M. (1983). Introduction to stochastic dynamic programming. New York, NY: Academic Press.
Santos, M., & Rust, J. (2004). Convergence properties of policy iteration. SIAM Journal on Control and Optimization, 42(6), 2094–2115.
Scherrer, B. (2013). Improved and generalized upper bounds on the complexity of policy iteration. In Advances in Neural Information Processing Systems, pp. 386–394
Shu, Z., & Jirutitijaroen, P. (2014). Optimal operation strategy of energy storage system for grid-connected wind power plants. IEEE Transactions on Sustainable Energy, 5(1), 190–199.
Sparacino, A., Reed, G., Kerestes, R., Grainger, B., & Smith, Z. (2012). Survey of battery energy storage systems and modeling techniques. In 2012 IEEE Power and Energy Society General Meeting, pp. 1–8
Topkis, D. (1998). Supermodularity and complementarity. Princeton: Princeton University Press.
Varah, J. (1975). A lower bound for the smallest singular value of a matrix. Linear Algebra and its Applications, 11, 3–5.
van de Ven, P., Hegde, N., Massoulié, L., & Salonidis, T. (2013). Optimal control of end-user energy storage. IEEE Transactions on Smart Grid, 4(2), 789–797.
Weron, R. (2014). Electricity price forecasting: A review of the state-of-the-art with a look into the future. International Journal of Forecasting, 30(4), 1030–1081.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hu, Y., Defourny, B. Optimal price-threshold control for battery operation with aging phenomenon: a quasiconvex optimization approach. Ann Oper Res 317, 623–650 (2022). https://doi.org/10.1007/s10479-017-2505-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-017-2505-4