Abstract
One of the goals of Smart Grids is to encourage distributed generation of energy in houses, hence allowing the user to profit by injecting energy into the power grid. The implementation of a differentiated tariff of energy per time of use, coupled with energy storage in batteries, enables profit maximization by the user, who can choose to sell or store the energy generated whenever it is convenient. This paper proposes a solution to the sequential decision-making problem of energy sale by applying reinforcement learning. Results show a significant increase in the total long-term profit by using the policy obtained with the proposed approach, when compared with a price-unaware selling policy.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Hammoudeh, M.A., Mancilla-David, F., Selman, J.D., Papantoni-Kazakos, P.: Comunication Architectures for Distribution Networks within the Smart Grid Initiative. In: Green Technologies Conference, IEEE, p 65,70 (2013), doi:10.1109/GreenTech.2013.18
Hashmi, M., Hanninen, S., Maki, K.: Survey of smart grid concepts, architectures, and technological demonstrations worldwide. In: IEEE PES Conference on Innovative Smart Grid Technologies (ISGT Latin America), p 1,7 (2011), doi:10.1109/ISGT-LA.2011.6083192
Uluski, R.W.: The role of Advanced Distribution Automation in the Smart Grid. In: Power and Energy Society General Meeting, IEEE, p 1,5 (2010), doi:10.1109/PES.2010.5590075
Palensky, P., Dietrich, D.: Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads. IEEE Trans. Ind. Inf. 7 (3), 381,388 (2011). doi:10.1109/TII.2011.2158841
Chen, C., Kishore, S., Zhifang, W., Alizadeh, M., Scaglione, A.: How will demand response aggregators affect electricity markets? A Cournot game analysis. In: 5th International Symposium on Communications Control and Signal Processing (ISCCSP), p 1,6 (2012), doi:10.1109/ISCCSP.2012.6217839
Parvania, M., Fotuhi-Firuzabad, M., Shahidehpour, M.: Demand response participation in wholesale energy markets. In: Power and Energy Society General Meeting, IEEE, p 1,4 (2012), doi:10.1109/PESGM.2012.6344591
Shishebori, A., Kian, A.R.: Risk analysis for distribution company energy procurement with pool market, DGs and demand response. In: 18th Iranian Conference on Electrical Engineering (ICEE), p 949,954 (2010), doi:10.1109/IRANIANCEE.2010.5506940
Chua-Liang, S., Kirschen, D.: Quantifying the Effect of Demand Response on Electricity Markets. IEEE Trans. Power Syst. 24 (3), 1199,1207 (2009). doi:10.1109/TPWRS.2009.2023259
Keen, P.: Decision support systems : a research perspective. Center for Information Systems Research, Cambridge (1980)
Wright, A., Sittig, D.: A framework and model for evaluating clinical decision support architectures q. J. Biomed. Inform. 41 (2008). doi:10.1016/j.jbi.2008.03.009
Power, D.J.: Decision support systems: concepts and resources for managers. Westport, Conn., Quorum Books (2002)
Stephens, W., Middleton, T.: Why has the uptake of Decision Support Systems been so poor?. In: Crop-soil simulation models in developing countries. Wallingford:CABI (2002)
Sol, H. G., Takkenberg, C. A.Th., de Vries Robb, P. F.: Expert systems and artificial intelligence in decision support systems. Second Mini Euroconference, Lunteren (1987). ISBN: 90-277-2437-7
Efraim, T., Jay, E.: Decision Support Systems and Intelligent Systems, p. 574. Prentice Hall (2008). ISBN-13: 978-0137409372
Dusparic, I., Harris, C., Marinescu, A., Cahill, V., Clarke, S.: Multi-agent residential demand response based on load forecasting. In: 1st IEEE Conference on Technologies for Sustainability (SusTech), p 90,96 (2013), doi:10.1109/SusTech.2013.6617303
O’Neill, D., Levorato, M., Goldsmith, A., Mitra, U.: Residential Demand Response Using Reinforcement Learning. In: First IEEE International Conference on Smart Grid Communications (SmartGridComm), p 409,414 (2010), doi:10.1109/SMARTGRID.2010.5622078
Chen, X., Wei, T., Hu, S.: Uncertainty-Aware Household Appliance Scheduling Considering Dynamic Electricity Pricing in Smart Home. IEEE Trans. Smart Grid 4 (2), 932,941 (2013). doi:10.1109/TSG.2012.2226065
Mohsenian-Rad, A.H., Wong, V.W.S., Jatskevich, J., Schober, R., Leon-Garcia, A.: Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid. IEEE Trans. Smart Grid 1 (3), 320,331 (2010). doi:10.1109/TSG.2010.2089069
Atzeni, I., Ordonez, L.G., Scutari, G., Palomar, D.P., Fonollosa, J.R.: Demand-side management via distributed energy generation and storage optimization. IEEE Trans. Smart Grid 4 (2), 866,876 (2013). doi:10.1109/TSG.2012.2206060
Balijepalli, V.S.K.M., Pradhan, V., Khaparde, S.A., Shereef, R.M.: Review of demand response under smart grid paradigm. In: Innovative Smart Grid Technologies - India (ISGT India), IEEE PES, p 236,243 (2011)
Albadi, M.H., El-Saadany, E.F.: Demand Response in Electricity Markets: An Overview. In: Power Engineering Society General Meeting, IEEE, p 1,5 (2007), doi:10.1109/PES.2007.385728
Carpinelli, G., Celli, G., Mocci, S., Mottola, F., Pilo, F., Proto, D.: Optimal Integration of Distributed Energy Storage Devices in Smart Grids. IEEE Trans. Smart Grid 4 (2), 985,995 (2013). doi:10.1109/TSG.2012.2231100
Zhimin, W., Chenghong, G., Furong, L., Bale, P., Hongbin, S.: Active Demand Response Using Shared Energy Storage for Household Energy Management. IEEE Trans. Smart Grid 4 (4), 1888,1897 (2013). doi:10.1109/TSG.2013.2258046
PJM monthly locational marginal pricing [Online]. Available: http://www.pjm.com/markets-and-operations/energy/real-time/monthlylmp.aspx
Bueno, E.A.B., Utubey, W., Hostt, R.R.: Evaluating the effect of the white tariff on a distribution expansion project in Brazil. In: IEEE PES Conference On Innovative Smart Grid Technologies Latin America (ISGT LA), p 1,8 (2013), doi:10.1109/ISGT-LA.2013.6554479
Russell, S.J., Norvig, P., 2nd edn: Artificial Intelligence: A Modern Approach. Pearson Education, Inc., Upper Saddle River (2003)
Sutton, R.S., Barto, A.G., Hu, S.: Reinforcement Learning: An Introduction. The MIT Press, Cambrige
Watkins, C.J.C.H.: Learning from Delayed Rewards, PhD thesis. Cambridge University, Cambridge (1989)
Kyocera Solar, Data Sheet of KD200-54 P Series PV Modules [On- line]. Available: http://www.kyocerasolar.com/assets/001/5124.pdf
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Berlink, H., Kagan, N. & Reali Costa, A.H. Intelligent Decision-Making for Smart Home Energy Management. J Intell Robot Syst 80 (Suppl 1), 331–354 (2015). https://doi.org/10.1007/s10846-014-0169-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-014-0169-8