Abstract
Artificial Intelligence (AI) powered building control allows deriving policies that are more flexible and energy efficient than standard control. However, there are challenges: environment interaction is used to train Reinforcement Learning (RL) agents but for building control it is often not possible to use a physical environment, and creating high fidelity simulators is a difficult task. With offline RL an agent can be trained without environment interaction, it is a data-driven approach to RL. In this paper, Conservative Q-Learning (CQL), an offline RL algorithm, is used to control the temperature setpoint in a room of a university campus building. The agent is trained using only the historical data available for this room. The results show that there is potential for offline RL in the field of building control, but also that there is room for improvement and need for further research in this area.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brandi, S., Piscitelli, M., Martellacci, M., Capozzoli, A.: Deep reinforcement learning to optimise indoor temperature control and heating energy consumption in buildings. Energy Build. 224, 110225 (2020)
Levine, S., Kumar, A., Tucker, G., Fu, J.: Offline reinforcement learning: tutorial, review, and perspectives on open problems (2020)
Kumar, A., Zhou, A., Tucker, G., Levine, S.: Conservative Q-Learning for Offline Reinforcement Learning (2020)
Wang, Z., et al.: Critic regularized regression (2020)
Wang, Z., Hong, T.: Reinforcement learning for building controls: the opportunities and challenges. Appl. Energy 269, 115036 (2020)
Ruelens, F., Claessens, B.J., Vandael, S., De Schutter, B., Babuska, R., Belmans, R.: Residential demand response of thermostatically controlled loads using batch reinforcement learning. IEEE Trans. Smart Grid 8(5), 2149–2159 (2017)
Zhang, Z., Chong, A., Pan, Y., Zhang, C., Lam, K.P.: Whole building energy model for HVAC optimal control: a practical framework based on deep reinforcement learning. Energy Build. 199, 472–490 (2019)
Dulac-Arnold, G., et al.: Deep Reinforcement Learning in Large Discrete Action Spaces. Technical Report, Google DeepMind (2015)
Han, M., et al.: A review of reinforcement learning methodologies for controlling occupant comfort in buildings. Sustain. Cities Soc. 51, 101748 (2019)
Afroz, Z., Shafiullah, G.M., Urmee, T., Higgins, G.: Modeling techniques used in building HVAC control systems: a review. Renew. Sustain. Ener. Rev. 83, 64–84 (2018)
Jia, R., Jin, M., Sun, K., Hong, T., Spanos, C.: Advanced building control via deep reinforcement learning. Energy Procedia 158, 6158–6163 (2019)
Monier, L., et al.: Offline Reinforcement Learning Hands-On (2020)
Seno, T.: d3rlpy: an offline deep reinforcement library (2020). https://github.com/takuseno/d3rlpy
Elmaz, F., Eyckerman, R., Casteels, W., Latré, S., Hellinckx, P.: CNN-LSTM Architecture for Predictive Indoor Temperature Modeling. Technical Report, IDLab - Faculty of Applied Engineering, Antwerp (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schepers, J., Eyckerman, R., Elmaz, F., Casteels, W., Latré, S., Hellinckx, P. (2022). Autonomous Building Control Using Offline Reinforcement Learning. In: Barolli, L. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing. 3PGCIC 2021. Lecture Notes in Networks and Systems, vol 343. Springer, Cham. https://doi.org/10.1007/978-3-030-89899-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-89899-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89898-4
Online ISBN: 978-3-030-89899-1
eBook Packages: EngineeringEngineering (R0)