Abstract
Due to the intermittent production of renewable energy and the time-varying power demand, microgrids (MGs) can exchange energy with each other to enhance their operational performance and reduce their dependence on power plants. In this paper, we investigate the energy trading game in smart grids, in which each MG chooses its energy trading strategy with its connected MGs and power plants according to the energy generation model, the current battery level, the energy demand, and the energy trading history. The Nash equilibria of this game are provided, revealing the conditions under which the MGs can satisfy their energy demands by using local renewable energy generations. In a dynamic version of the game, a Q-learning based strategy is proposed for an MG to obtain the optimal energy trading strategy with other MGs and the energy plants without being aware of the future energy consumption model and the renewable generation of other MGs in the trading market. We apply the estimated renewable energy generation model of the MG and design a hotbooting technique to exploit the energy trading experiences in similar scenarios to initialize the quality values in the learning process to accelerate the convergence speed. The proposed hotbooting Q-learning based energy trading scheme significantly reduces the total energy that the MGs in the smart grid purchase from the power plant and improves the utility of the MG.
This research was supported in part by National Natural Science Foundation of China under Grant 61671396 and the CCF-Venustech Hongyan Research Initiative (2016-010).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Amin, S.M., Wollenberg, B.F.: Toward a smart grid: power delivery for the 21st century. IEEE Trans. Smart Grid 3(5), 34–41 (2005)
Farhangi, H.: The path of the smart grid. IEEE Power Energ. Mag. 8(1), 18–28 (2010)
Baeyens, E., Bitar, E., Khargonekar, P.P., Poolla, K.: Wind energy aggregation: a coalitional game approach. In: Decision and Control and European Control Conference, pp. 3000–3007 (2011)
Maharjan, S., Zhu, Q., Zhang, Y., Gjessing, S., Basar, T.: Dependable demand response management in the smart grid: a stackelberg game approach. IEEE Trans. Smart Grid 4(1), 120–132 (2013)
Wang, Y., Saad, W., Han, Z., Poor, H.V., Basar, T.: A game-theoretic approach to energy trading in the smart grid. IEEE Trans. Smart Grid 5(3), 1439–1450 (2014)
Tushar, W., Chai, B., Yuen, C., et al.: Three-party energy management with distributed energy resources in smart grid. IEEE Trans. Ind. Electron. 62(4), 2487–2498 (2015)
Xiao, L., Mandayam, N.B., Poor, H.V.: Prospect theoretic analysis of energy exchange among microgrids. IEEE Trans. Smart Grid 6(1), 63–72 (2015)
Xiao, L., Chen, Y., Liu, K.R.: Anti-cheating Prosumer Energy Exchange based on Indirect Reciprocity. In: IEEE International Conference on Communication, pp. 599–604. Sydney (2014)
Guan, C., Wang, Y., Lin, X., Nazarian, S., Pedram, M.: Reinforcement learning-based control of residential energy storage systems for electric bill minimization. In: IEEE Consumer Communication and Networking Conference, pp. 637–642. Las Vegas (2015)
Qiu, X., Nguyen, T.A., Crow, M.L.: Heterogeneous energy storage optimization for microgrids. IEEE Trans. Smart Grid 7(4), 1453–1461 (2016)
Dalal, G., Gilboa, E., Mannor, S.: Hierarchical decision making in electricity grid management. In: International Conference on Machine Learning, New York, pp. 2197–2206 (2016)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1966)
Wang, H., Huang, J.: Incentivizing Energy Trading for Interconnected Microgrids. IEEE Trans. Smart Grid (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Xiao, X., Dai, C., Li, Y., Zhou, C., Xiao, L. (2017). Energy Trading Game for Microgrids Using Reinforcement Learning. In: Duan, L., Sanjab, A., Li, H., Chen, X., Materassi, D., Elazouzi, R. (eds) Game Theory for Networks. GameNets 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 212. Springer, Cham. https://doi.org/10.1007/978-3-319-67540-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-67540-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67539-8
Online ISBN: 978-3-319-67540-4
eBook Packages: Computer ScienceComputer Science (R0)