Skip to main content

Multi-zone Residential HVAC Control with Satisfying Occupants’ Thermal Comfort Requirements and Saving Energy via Reinforcement Learning

  • Conference paper
  • First Online:
Parallel and Distributed Computing, Applications and Technologies (PDCAT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13148))

Abstract

Residential HVAC system control has been focused on thermal comfort and energy consumption. Due to the complexity of the dynamic building thermal model, weather conditions and human activities, traditional methods such as rule-based control (RBC) and model predictive control (MPC) are difficult to learn a strategy that can save energy while satisfying occupants’ thermal comfort requirements. To solve the above problem, we propose a method combining a thermal comfort prediction model and reinforcement learning to optimize residential multi-zone HVAC control. In this paper, we first design a hybrid model of Support Vector Regression and a Deep Neural Network (SVR-DNN) to predict thermal comfort value, which is taken as a part of the state and reward in reinforcement learning. Then we apply reinforcement learning algorithms (Q-learning, Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG)) to respectively generate an optimal HVAC control strategy to maintain the stability of thermal comfort and minimize energy consumption. The experimental results show that our SVR-DNN model can improve thermal comfort prediction performance by \(20.5\%\) compared with the deep neural network (DNN); compared with rule-based control, DDPG, DQN and Q-learning based on SVR-DNN can reduce energy consumption by \(11.89\%\), \(8.41\%\), \(6.51\%\) and reduce thermal comfort violation by \(91.8\%\), \(43.2\%\), \(25.4\%\).

This work was financially supported by Primary Research and Development Plan of China (No. 2020YFC2006602), National Natural Science Foundation of China (No. 62072324, No. 61876217, No. 61876121, No. 61772357), University Natural Science Foundation of Jiangsu Province (No. 21KJA520005), Primary Research and Development Plan of Jiangsu Province (No. BE2020026), Natural Science Foundation of Jiangsu Province (No. BK20190942).

The code and data are available at https://github.com/DZKK1234/Multi-zone-residential-HVAC-control.git.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Pérez-Lombard, L., Ortiz, J., Pout, C.: A review on buildings energy consumption information. Energy Build. 40(3), 394–398 (2008). https://doi.org/10.1016/j.enbuild.2007.03.007

    Article  Google Scholar 

  2. Shepherd, A., Batty, W.: Fuzzy control strategies to provide cost and energy efficient high quality indoor environments in buildings with high occupant densities. Build. Serv. Eng. Res. Technol. 24(1), 35–45 (2003)

    Article  Google Scholar 

  3. Qiu, S., Li, Z., Li, Z., Li, J., Long, S., Li, X.: Model-free control method based on reinforcement learning for building cooling water systems: Validation by measured data-based simulation. Energy Build. 218, 110055 (2020). https://doi.org/10.1016/j.enbuild.2020.110055

    Article  Google Scholar 

  4. Brandi, S., Piscitelli, M.S., Martellacci, M., Capozzoli, A.: Deep reinforcement learning to optimise indoor temperature control and heating energy consumption in buildings. Energy Build. 224, 110225 (2020)

    Article  Google Scholar 

  5. Jiang, Z., et al.: Building HVAC control with reinforcement learning for reduction of energy cost and demand charge. Energy Build. 239, 110833 (2021)

    Article  Google Scholar 

  6. Shaw, E.W.: Thermal comfort: analysis and applications in environmental engineering, by p. o. fanger. 244 pp. danish technical press. copenhagen, denmark, 1970. danish kr. 76, 50. J. Roy. Soc. Promot. Health (1972)

    Google Scholar 

  7. Zhou, X., et al.: Data-driven thermal comfort model via support vector machine algorithms: insights from ashrae rp-884 database. Energy Build. 211, 109795 (2020). https://doi.org/10.1016/j.enbuild.2020.109795

    Article  Google Scholar 

  8. Liu, W., Lian, Z., Zhao, B.: A neural network evaluation model for individual thermal comfort. Energy Build. 39(10), 1115–1122 (2007)

    Article  Google Scholar 

  9. Wei, T., Zhu, Q., Maasoumy, M.: Co-scheduling of HVAC control, EV charging and battery usage for building energy efficiency. In: Chang, Y. (ed.) The IEEE/ACM International Conference on Computer-Aided Design, ICCAD 2014, San Jose, CA, USA, November 3–6, 2014, pp. 191–196. IEEE (2014)

    Google Scholar 

  10. Zenger, A., Schmidt, J., Krödel, M.: Towards the intelligent home: using reinforcement-learning for optimal heating control. In: Timm, I.J., Thimm, M. (eds.) KI 2013. LNCS (LNAI), vol. 8077, pp. 304–307. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40942-4_30

    Chapter  Google Scholar 

  11. Du, Y., et al.: Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning. Appl. Energy 281, 116117 (2021)

    Article  Google Scholar 

  12. Deng, J., Yao, R., Yu, W., Zhang, Q., Li, B.: Effectiveness of the thermal mass of external walls on residential buildings for part-time part-space heating and cooling using the state-space method. Energy Build. 190, 155–171 (2019). https://doi.org/10.1016/j.enbuild.2019.02.029

    Article  Google Scholar 

  13. Bureau, C.M., et al.: China Standard Weather Data for Analyzing Building Thermal Conditions, pp. 90–105. China Building Industry Publishing House, Beijing (2005)

    Google Scholar 

  14. Földváry Ličina, V., et al.: Development of the ashrae global thermal comfort database II. Build. Environ. 142, 502–512 (2018). https://doi.org/10.1016/j.buildenv.2018.06.022

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Qiming Fu or Jianping Chen .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ding, Z., Fu, Q., Chen, J., Wu, H., Lu, Y., Hu, F. (2022). Multi-zone Residential HVAC Control with Satisfying Occupants’ Thermal Comfort Requirements and Saving Energy via Reinforcement Learning. In: Shen, H., et al. Parallel and Distributed Computing, Applications and Technologies. PDCAT 2021. Lecture Notes in Computer Science(), vol 13148. Springer, Cham. https://doi.org/10.1007/978-3-030-96772-7_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-96772-7_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-96771-0

  • Online ISBN: 978-3-030-96772-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics