Abstract
Demand Response (DR) has become a key strategy for enhancing energy system sustainability and reducing costs. Deep Learning (DL) has emerged as crucial for managing DR's complexity and large data volumes, enabling near real-time decision-making. DL techniques can effectively tackle challenges such as selecting responsive users, understanding consumption behaviours, optimizing pricing, monitoring and controlling devices, engaging more consumers in DR schemes, and determining fair remuneration for participants. This research work presents an integrated architecture for smart grid energy management, combining a Deep Attention-Enhanced Sequence-to-Sequence Model (AES2S) with Energy-Aware Optimized Reinforcement Learning (EAORL). The objective is to design a system that performs non-intrusive load monitoring and optimizes demand response to enhance energy efficiency while maintaining user comfort. The AES2S module accurately performs appliance state identification and load disaggregation using convolutional layers, Enhanced Sequence-to-Sequence Model networks, and an attention mechanism. The EAORL module employs a multi-agent system, where each agent uses a Deep Q-Learning Network to learn optimal policies for adjusting energy consumption in response to grid conditions and user demand. The system uses an Iterative Policy Update mechanism, where agents update their policies sequentially, ensuring stable and effective learning. The integration ensures seamless data flow, with AES2S outputs enhancing EAORL state representations. Validated in a simulated smart grid environment, the architecture dynamically adjusts energy consumption, demonstrating significant improvements in energy efficiency, cost reduction, and user comfort. Evaluation metrics confirm the system's effectiveness, making AES2S-EAORL a robust solution for smart grid energy management and demand response optimization.









Similar content being viewed by others
Data Availability
The dataset produced and scrutinized in this study are accessible from the corresponding author upon reasonable request.
References
Mahmud K, Ravishankar J, Hossain MJ, Dong ZY. The impact of prediction errors in the domestic peak power demand management. IEEE Trans Ind Inf. 2020;16(7):4567–79.
Tao Y, Qiu J, Lai S, Zhao J, Xue Y. Carbon-oriented electricity network planning and transformation. IEEE Trans Power Syst. 2021;36(2):1034–48.
Velasquez W, Moreira-Moreira GZ, Alvarez-Alvarado MS. Smart grids empowered by software-defined network: a comprehensive review of advancements and challenges. IEEE Access. 2024;12:63400–16. https://doi.org/10.1109/ACCESS.2024.3396402.
Shafie-khah M, Siano P, Aghaei J, Masoum MAS, Li F, Catalão JPS. Comprehensive review of the recent advances in industrial and commercial DR. IEEE Trans Ind Inf. 2019;15(7):3757–71.
Bucher D, et al. Incentivising demand side response through discount scheduling using hybrid quantum optimization. IEEE Trans Quantum Eng. 2024;5:1–15. https://doi.org/10.1109/TQE.2024.3407236.
Deltetto D, Coraci D, Pinto G, Piscitelli MS, Capozzoli A. Exploring the potentialities of deep reinforcement learning for incentive-based demand response in a cluster of small commercial buildings. Energies. 2021;14(10):2933.
He B, Zhang N, Fang C, Su Y, Wang Y. Flexible building energy management with neural ODEs-based model predictive control. IEEE Trans Smart Grid. 2024;15(5):4690–704. https://doi.org/10.1109/TSG.2024.3401227.
Pereira H, Faia R, Gomes L, Faria P, Vale Z. Congestion management in energy communities using a two-step demand response methodology. IEEE Trans Ind Appl. 2024;60(4):5344–51. https://doi.org/10.1109/TIA.2024.3395228.
Khan MA, Saleh AM, Waseem M, Sajjad IA. Artificial intelligence enabled demand response: prospects and challenges in smart grid environment. IEEE Access. 2023;11:1477–505. https://doi.org/10.1109/ACCESS.2022.3231444.
Yadav RK, Hrisheekesha PN, Bhadoria VS. Grey Wolf optimization based demand side management in solar PV integrated smart grid environment. IEEE Access. 2023;11:11827–39. https://doi.org/10.1109/ACCESS.2023.3241856.
Liu Y, Qiu B, Fan X, Zhu H, Han B. Review of smart home energy management systems. Energy Proc. 2016;104:504–8.
Beaudin M, Zareipour H. Home energy management systems: a review of modelling and complexity. Renew Sustain Energy Rev. 2015;45:318–35.
Zhou B, Li W, Chan KW, Cao Y, Kuang Y, Liu X, Wang X. Smart home energy management systems: concept, configurations, and scheduling strategies. Renew Sustain Energy Rev. 2016;61:30–40.
Ahmad A, Khan A, Javaid N, Hussain HM, Abdul W, Almogren A, Alamri A, Niaz IA. An optimized home energy management system with integrated renewable energy and storage resources. Energies. 2017;10(4):549.
Mahapatra B, Nayyar A. Home energy management system (HEMS): concept, architecture, infrastructure, challenges and energy management schemes. Energy Syst. 2022;13(3):643–69.
Deepa KR, Kodabagi MM, Ravishankar H, Sanjana G, Dhruva Kumari S, Rachana KJ. Intelligent learning methods for load frequency control in smart homes. In: 2023 IEEE 2nd international conference on industrial electronics: developments & applications (ICIDeA), Imphal, India, pp 316–321; 2023. https://doi.org/10.1109/ICIDeA59866.2023.10295181.
Waseem M, Lin Z, Liu S, Sajjad IA, Aziz T. Optimal GWCSO based home appliances scheduling for demand response considering end-users comfort. Electr Power Syst Res. 2020;187: 106477.
Lee S, Choi D-H. Energy management of smart home with home appliances, energy storage system and electric vehicle: a hierarchical deep reinforcement learning approach. Sensors. 2020;20(7):2157.
Lissa P, Deane C, Schukat M, Seri F, Keane M, Barrett E. Deep reinforcement learning for home energy management system control. Energy AI. 2021;3: 100043.
Tantawy ESF, Amer GM, Fayez HM. Scheduling home appliances with integration of hybrid energy sources using intelligent algorithms. Ain Shams Eng J. 2022;13(4): 101676.
Murray D, Stankovic L, Stankovic V. An electrical load measurements dataset of United Kingdom households from a two-year longitudinal study. Sci data. 2017;4(1):1–12. https://doi.org/10.1038/sdata.2016.122.
Kelly J, Knottenbelt W. Theuk-dale dataset, domestic appliance level electricity demand and whole-house demand from five UK homes. Sci data. 2015;2(1):1–14.
Tanoni G, Principi E, Squartini S. Multi-label appliance classification with weakly labeled data for non-intrusive load monitoring. IEEE Trans Smart Grid. 2022;14(1):440–52.
Langevin A, Carbonneau M-A, Cheriet M, Gagnon G. LD using variational autoencoders. Energy Build. 2022;254: 111623.
Pirnat A, Bertalanič B, Cerar G, Mohorčič M, Fortuna C. Energy efficient deep multi-label ON/OFF classification of low frequency metered home appliances. IEEE Access. 2024;12:51966–81. https://doi.org/10.1109/ACCESS.2024.3382830.
Funding
No funding involved in the research work.
Author information
Authors and Affiliations
Contributions
The research resulted from a collective effort, with all authors contributing collaboratively to its accomplishment.
Corresponding author
Ethics declarations
Conflicts of Interest
Authors have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Deepa, K.R., Thillaiarasu, N. Integrated Architecture for Smart Grid Energy Management: Deep Attention-Enhanced Sequence-to-Sequence Model with Energy-Aware Optimized Reinforcement Learning for Demand Response. SN COMPUT. SCI. 5, 1017 (2024). https://doi.org/10.1007/s42979-024-03305-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-024-03305-2