Abstract
Grid-convenience is often seen as conflicting with other measures of interest, such as self-use or resilience. We adopt a Hybrid Petri net model (HPnG) of a smart home with local power generation, local storage and different battery management strategies in the presence of power outages. Applying Q-learning allows us to derive schedulers with an optimal loading percentages for the battery. We show that (near-)optimal schedulers can be synthesized for accurate predictions, which achieve grid-convenience without decreasing resilience and self-use. Introducing uncertainty to the predictions of solar production shows that a good balance can be maintained between such measures, when allowing the system to adapt the loading percentage during the day.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Cai, M., Peng, H., Li, Z., Kan, Z.: Learning-based probabilistic LTL motion planning with environment and motion uncertainties. IEEE Trans. Autom. Control 66(5), 2386–2392 (2021)
del Real, A.J., Dorado, F., Durán, J.: Energy demand forecasting using deep learning: applications for the French grid. Energies 13(9), 2242 (2020)
EDSN: EDSN demand profiles. http://www.edsn.nl/verbruiksprofielen/
Fulton, N., Hunt, N., Hoang, N., Das, S.: Formal verification of end-to-end learning in cyber-physical systems: progress and challenges. arXiv:2006.09181 (2020)
Gasser, P., et al.: A review on resilience assessment of energy systems. Sustain. Resilient Infrastruct. 6(5), 1–27 (2019)
Ghasemieh, H., Haverkort, B.R., Jongerden, M.R., Remke, A.: Energy resilience modelling for smart houses. In: Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015), pp. 275–286 (2015)
Gieseke, F., Igel, C.: Training big random forests with little resources. In: 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1445–1454. ACM (2018)
Hahn, E.M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., Wojtczak, D.: Faithful and effective reward schemes for model-free reinforcement learning of omega-regular objectives. In: Hung, D.V., Sokolsky, O. (eds.) ATVA 2020. LNCS, vol. 12302, pp. 108–124. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59152-6_6
Hasanbeig, M., Kantaros, Y., Abate, A., Kroening, D., Pappas, G.J., Lee, I.: Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In: IEEE Conference on Decision and Control (CDC), pp. 5338–5343. IEEE, Nice, France (2019)
Hasanbeig, M., Abate, A., Kroening, D.: Cautious reinforcement learning with logical constraints. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020, Auckland, New Zealand, 9–13 May 2020, pp. 483–491. International Foundation for Autonomous Agents and Multiagent Systems (2020)
Huels, J., Remke, A.: Energy storage in smart homes: grid-convenience versus self-use and survivability. In: 24th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, MASCOTS 2016, London, United Kingdom, 19–21 September 2016, pp. 385–390 (2016)
Jongerden, M., Hüls, J., Remke, A., Haverkort, B.R.: Does your domestic photovoltaic energy system survive grid outages? Energies 9(9), 736:1–736:17 (2016)
Junges, S., Jansen, N., Katoen, J.-P., Topcu, U., Zhang, R., Hayhoe, M.: Model checking for safe navigation among humans. In: McIver, A., Horvath, A. (eds.) QEST 2018. LNCS, vol. 11024, pp. 207–222. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99154-2_13
Lavaei, A., Somenzi, F., Soudjani, S., Trivedi, A., Zamani, M.: Formal controller synthesis for continuous-space MDPs via model-free reinforcement learning. In: 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS), pp. 98–107. IEEE, Sydney, Australia (2020)
Mola, M., Feofilovs, M., Romagnoli, F.: Energy resilience: research trends at urban, municipal and country levels. Energy Procedia 147, 104–113 (2018)
Niehage, M., Hartmanns, A., Remke, A.: Learning optimal decisions for stochastic hybrid systems. In: 19th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2021. ACM (2021)
Niehage, M., Remke, A.: Learning that grid-convenience does not hurt resilience in the presence of uncertainty (artifact) (2022). https://doi.org/10.5281/zenodo.6840881
NREL: PVWatts. http://pvwatts.nrel.gov/index.php
Pilch, C., Edenfeld, F., Remke, A.: HYPEG: statistical model checking for hybrid petri nets: tool paper. In: EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), pp. 186–191. ACM Press (2017)
Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: 2017 47th IEEE/IFIP International Conference on Dependable Systems & Networks, DSN, pp. 475–486. IEEE (2017)
Pilch, C., Remke, A.: Statistical model checking for hybrid petri nets with multiple general transitions. In: Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 475–486. IEEE (2017)
Sadigh, D., Kim, E.S., Coogan, S., Sastry, S.S., Seshia, S.A.: A learning based approach to control synthesis of Markov decision processes for linear temporal logic specifications. In: IEEE Conference on Decision and Control, pp. 1091–1096. IEEE (2014)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series, 2nd edn. The MIT Press, Cambridge (2018)
Tagawa, Y., et al.: Day-ahead scheduling for supply-demand-storage balancing - model predictive generation with interval prediction of photovoltaics. In: 2015 European Control Conference (ECC), pp. 247–252 (2015)
Yu, L., et al.: Deep reinforcement learning for smart home energy management. IEEE Internet Things J. 7(4), 2751–2762 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Niehage, M., Remke, A. (2022). Learning that Grid-Convenience Does Not Hurt Resilience in the Presence of Uncertainty. In: Bogomolov, S., Parker, D. (eds) Formal Modeling and Analysis of Timed Systems. FORMATS 2022. Lecture Notes in Computer Science, vol 13465. Springer, Cham. https://doi.org/10.1007/978-3-031-15839-1_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-15839-1_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15838-4
Online ISBN: 978-3-031-15839-1
eBook Packages: Computer ScienceComputer Science (R0)