Skip to main content

Advertisement

SpringerLink
Log in

Application of deep reinforcement learning to intelligent distributed humidity control system

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The indoor environment of buildings is complex and changeable, and it is difficult to ensure that the indoor humidity is uniform and stable while employing a centralized humidity control system. To address this challenge, this paper proposes an intelligent distributed humidity control system based on model-free deep reinforcement learning. The proposed system consists of three parts: an intelligent controller, distributed facilities, and distributed sensors. The distributed sensors are used to monitor the environmental parameters. This study developed a reinforcement learning algorithm called RH-rainbow and deployed it in distributed facilities. In RH-rainbow, the reward consists of the mean absolute difference of humidity and the energy consumption of distributed facilities. The action is the humidity setpoints and fan settings of the constant humidity machines. The performance of RH-rainbow was evaluated and compared to that of other algorithms in two scenarios with different air outlet settings under different sensor numbers, reporting time intervals, and external interference modes. It was found that RH-rainbow is superior to manual strategies, the traditional analog control strategy, DQN, and PID in terms of uniformity, anti-interference ability, and energy consumption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. AIC Wiki (2020) Accessed 12 May 2020 Environmental guidelines. https://www.conservation-wiki.com/wiki/Environmental_Guidelines

  2. Atkinson JK (2014) Environmental conditions for the safeguarding of collections: a background to the current debate on the control of relative humidity and temperature. Stud Conserv 59(4):205–212. https://doi.org/10.1179/2047058414Y.0000000141

    Article  Google Scholar 

  3. Plenderleith H, Philippot P (1960) Climatologie et conservation dans les musées: sommaire. Museum Int (Edition Francaise) 13(4):201–289. https://doi.org/10.1111/j.1755-5825.1960.tb01558.x

    Article  Google Scholar 

  4. Ahn KU, Park CS (2020) Application of deep q-networks for model-free optimal control balancing between different hvac systems. Sci Technol Built Environ 26(1):61–74. https://doi.org/10.1080/23744731.2019.1680234

    Article  Google Scholar 

  5. Nagarathinam S, Menon V, Vasan A et al (2020) Marco-multi-agent reinforcement learning based control of building hvac systems. In: Proceedings of the eleventh ACM international conference on future energy systems, pp 57–67. https://doi.org/10.1145/3396851.3397694

  6. Zhao Y, Zhao Q, Xia L et al (2013) A unified control framework of hvac system for thermal and acoustic comforts in office building. In: 2013 IEEE international conference on automation science and engineering (CASE). IEEE, pp 416–421. https://doi.org/10.1109/CoASE.2013.6653964

  7. Ding X, Du W, Cerpa A (2019) Octopus: deep reinforcement learning for holistic smart building control. In: Proceedings of the 6th ACM international conference on systems for energy-efficient buildings, cities, and transportation, pp 326–335. https://doi.org/10.1145/3360322.3360857

  8. Wei T, Wang Y, Zhu Q (2017) Deep reinforcement learning for building hvac control. In: Proceedings of the 54th annual design automation conference 2017, pp 1–6. https://doi.org/10.1145/3061639.3062224

  9. Shuang X, Dongyang Z, Zhen L et al (2019) A combined control method of temperature and humidity inside the museum cabinet. In: 2019 11th International conference on measuring technology and mechatronics automation (ICMTMA). IEEE, pp 322–326. https://doi.org/10.1109/ICMTMA.2019.00078

  10. Xianzhe H (2011) Room temperature and humidity monitoring and energy-saving system. In: 2011 6th International conference on computer science & education (ICCSE). IEEE, pp 537–540. https://doi.org/10.1109/ICCSE.2011.6028696

  11. Zhao X, Tang J, Chen D (2009) Research of temperature and humidity decoupling control for central air-conditioning system. In: 2009 International conference on intelligent human-machine systems and cybernetics. IEEE, pp 404–408. https://doi.org/10.1109/IHMSC.2009.224

  12. Liu S, Wang X, Li S (2016) Fuzzy pid controller design of air handling unit for constant temperature and humidity air-conditioning. In: 2016 8th International conference on intelligent human-machine systems and cybernetics (IHMSC). IEEE, pp 410–414. https://doi.org/10.1109/IHMSC.2016.219

  13. Wu Q, Cai W, Shen S et al (2017) Dynamic analysis of an energy efficiency dehumidifier for building applications. In: 2017 12th IEEE conference on industrial electronics and applications (ICIEA). IEEE, pp 2060–2065. https://doi.org/10.1109/ICIEA.2017.8283177

  14. Wang X, Lu J, Yang Q et al (2013) Performance evaluation of packed tower liquid desiccant dehumidifier based on lssvm. In: 2013 10th Ieee international conference on control and automation (Icca). IEEE, pp 987–990. https://doi.org/10.1109/ICCA.2013.6565050

  15. Yuan X, Pan Y, Yang J et al (2019) Study on the application of reinforcement learning in the operation optimization of hvac system. Build Simul 14:75–87. https://doi.org/10.1007/s12273-020-0602-9

    Article  Google Scholar 

  16. Baghaee S, Ulusoy I (2018) User comfort and energy efficiency in hvac systems by q-learning. In: 2018 26th Signal processing and communications applications conference (SIU). IEEE, pp 1–4. https://doi.org/10.1109/SIU.2018.8404287

  17. Chen Y, Norford LK, Samuelson HW et al (2018) Optimal control of hvac and window systems for natural ventilation through reinforcement learning. Energy Build 169:195–205. https://doi.org/10.1016/j.enbuild.2018.03.051

    Article  Google Scholar 

  18. Qiu S, Li Z, Li Z et al (2020) Model-free control method based on reinforcement learning for building cooling water systems: validation by measured data-based simulation. Energy Build 218:110,055. https://doi.org/10.1016/j.enbuild.2020.110055

    Article  Google Scholar 

  19. Faddel S, Tian G, Zhou Q et al (2020) Data driven q-learning for commercial hvac control. In: 2020 SoutheastCon. IEEE, pp 1–6. https://doi.org/10.1016/10.1109/SoutheastCon44009.2020.9249737

  20. Biemann M, Scheller F, Liu X et al (2021) Experimental evaluation of model-free reinforcement learning algorithms for continuous hvac control. Appl Energy 298:117,164. https://doi.org/10.1016/j.apenergy.2021.117164

    Article  Google Scholar 

  21. Du Y, Zandi H, Kotevska O et al (2021) Intelligent multi-zone residential hvac control strategy based on deep reinforcement learning. Appl Energy 281:116,117. https://doi.org/10.1016/j.apenergy.2020.116117

    Article  Google Scholar 

  22. Gao G, Li J, Wen Y (2020) Deepcomfort: energy-efficient thermal comfort control in buildings via reinforcement learning. IEEE Internet Things J 7(9):8472–8484. https://doi.org/10.1109/JIOT.2020.2992117

    Article  Google Scholar 

  23. Valladares W, Galindo M, Gutiérrez J et al (2019) Energy optimization associated with thermal comfort and indoor air control via a deep reinforcement learning algorithm. Build Environ 155:105–117. https://doi.org/10.1016/j.buildenv.2019.03.038

    Article  Google Scholar 

  24. Sakuma Y, Nishi H (2020) Airflow direction control of air conditioners using deep reinforcement learning. In: 2020 SICE international symposium on control systems (SICE ISCS). IEEE, pp 61–68. https://doi.org/10.23919/SICEISCS48470.2020.9083565

  25. Yu L, Sun Y, Xu Z et al (2020) Multi-agent deep reinforcement learning for hvac control in commercial buildings. IEEE Trans Smart Grid 12(1):407–419. https://doi.org/10.1109/TSG.2020.3011739

    Article  Google Scholar 

  26. Xu S, Wang Y, Wang Y et al (2020) One for many: transfer learning for building hvac control. In: Proceedings of the 7th ACM international conference on systems for energy-efficient buildings, cities, and transportation, pp 230–239. https://doi.org/10.1145/3408308.3427617

  27. Zhang Y, Zhou Y, Lu H et al (2021) Cooperative multi-agent actor–critic control of traffic network flow based on edge computing. Futur Gener Comput Syst 123:128–141. https://doi.org/10.1016/j.future.2021.04.018

    Article  Google Scholar 

  28. Shang M, Zhou Y, Fujita H (2021) Deep reinforcement learning with reference system to handle constraints for energy-efficient train control. Inf Sci 570:708–721. https://doi.org/10.1016/j.ins.2021.04.088

    Article  MathSciNet  Google Scholar 

  29. Pérez S, Arroba P, Moya JM (2021) Energy-conscious optimization of edge computing through deep reinforcement learning and two-phase immersion cooling. Futur Gener Comput Syst 125:891–907. https://doi.org/10.1016/j.future.2021.07.031

    Article  Google Scholar 

  30. Deng X, Zhang Y, Zhang Y et al (2022) Towards optimal hvac control in non-stationary building environments combining active change detection and deep reinforcement learning. Build Environ 211:108,680. https://doi.org/10.1016/j.buildenv.2021.108680

    Article  Google Scholar 

  31. Tsenis TT, Kapsimanis G, Kappatos V (2021) Smartclima: reinforcement learning residential thermostat-less heating control system. In: 2021 International conference on electrical, computer, communications and mechatronics engineering (ICECCME), pp 1–6. https://doi.org/10.1109/ICECCME52200.2021.9591000

  32. Ferdyn-Grygierek J, Grygierek K (2019) Hvac control methods for drastically improved hygrothermal museum microclimates in warm season. Build Environ 149:90–99. https://doi.org/10.1016/j.buildenv.2018.12.018

    Article  Google Scholar 

  33. Hessel M, Modayil J, Van Hasselt H et al (2017) Rainbow: combining improvements in deep reinforcement learning. arXiv:1710.02298

  34. Gupta A, Badr Y, Negahban A et al (2021) Energy-efficient heating control for smart buildings with deep reinforcement learning. J Build Eng 34:101,739. https://doi.org/10.1016/j.jobe.2020.101739

    Article  Google Scholar 

  35. Schaul T, Quan J, Antonoglou I et al (2015) Prioritized experience replay. Comput Sci

  36. Lillicrap TP, Hunt JJ, Pritzel A et al (2015) Continuous control with deep reinforcement learning. arXiv:1509.02971

  37. Rockafellar RT (2015) Convex analysis. Princeton University Press. https://doi.org/10.1515/9781400873173

  38. Enriko IKA, Putra RA, Estananto (2021) Automatic temperature control system on smart poultry farm using pid method. Green Intell Syst Appl 1(1):37–43. https://doi.org/10.53623/gisa.v1i1.40

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers for their detailed reviews and constructive comments, which have helped improve the quality of this paper. And thank professor Zhu of Dalian University of Technology for providing simulation support. This work is supported by National Key R&D Program of China under Grant No. 2020YFC1522503.

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Work Safety Intelligent Monitoring, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Da Guo & Yong Zhang

  2. School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Danfeng Luo & Yunqi Sun

  3. Beijing PengTongGaoKe Technologies Co.,Ltd, City, 100068, China

    Xiuyong Zhang

  4. Ningbo Shuyou Technologies Co., Ltd, Street, Ningbo, 315104, China

    Yuyang Lai

Authors
  1. Da Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danfeng Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuyong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuyang Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunqi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, D., Luo, D., Zhang, Y. et al. Application of deep reinforcement learning to intelligent distributed humidity control system. Appl Intell 53, 16724–16746 (2023). https://doi.org/10.1007/s10489-022-04320-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04320-7

Keywords

Search

Navigation