Abstract
Natural gas load forecasting is essential to retailers in terms of profit-making and service quality. In practice, a retailer has limited consumer load data to build an accurate prediction model. Federated learning enables retailers to train a global model collaboratively, without compromising data privacy. However, it could not behave well on all consumers due to their diversity, e.g., different load patterns. To address this data heterogeneity issue, we propose two-phased federated learning with cluster-based personalization (CPFL) for natural gas load forecasting. Firstly, a knowledge-based federated clustering is proposed to categorize similar consumers from different retailers into clusters in a privacy-preserving manner. Then, vanilla federated learning is adopted to pre-train a global model, leveraging all available data from retailers. Finally, the pre-trained model is fine-tuned and personalized to each cluster respectively, using an attention-based model aggregation strategy according to the contribution difference of individual consumers in the cluster. Comprehensive experiments are conducted using a real-world data set with 2000 consumers from eight retailers, and the results show our proposed CPFL framework outperforms the state-of-the-art personalized federated learning approaches for time-series forecasting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Briggs, C., Fan, Z., Andras, P.: Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–9. IEEE (2020)
Chen, Y., Qin, X., Wang, J., Yu, C., Gao, W.: FedHealth: a federated transfer learning framework for wearable healthcare. IEEE Intell. Syst. 35(4), 83–93 (2020)
Fekri, M.N., Grolinger, K., Mir, S.: Distributed load forecasting using smart meter data: federated learning with recurrent neural networks. Int. J. Electr. Power Energy Syst. 137, 107669 (2022)
Guo, S., Li, Z., Liu, H., Zhao, S., Jin, C.H.: Personalized federated learning for multi-task fault diagnosis of rotating machinery. arXiv preprint arXiv:2211.09406 (2022)
Huang, L., Shea, A.L., Qian, H., Masurkar, A., Deng, H., Liu, D.: Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records. J. Biomed. Inform. 99, 103291 (2019)
Husnoo, M.A., Anwar, A., Hosseinzadeh, N., Islam, S.N., Mahmood, A.N., Doss, R.: FedREP: towards horizontal federated load forecasting for retail energy providers. arXiv preprint arXiv:2203.00219 (2022)
Ji, S., Pan, S., Long, G., Li, X., Jiang, J., Huang, Z.: Learning private neural language modeling with attentive aggregation. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: SCAFFOLD: stochastic controlled averaging for federated learning. In: International Conference on Machine Learning, pp. 5132–5143. PMLR (2020)
Li, P., Zhang, J.S.: A new hybrid method for China’s energy supply security forecasting based on ARIMA and XGBoost. Energies 11(7), 1687 (2018)
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)
Liu, J., Wang, S., Wei, N., Chen, X., Xie, H., Wang, J.: Natural gas consumption forecasting: a discussion on forecasting history and future challenges. J. Nat. Gas Sci. Eng. 90, 103930 (2021)
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Peng, S., Chen, R., Yu, B., Xiang, M., Lin, X., Liu, E.: Daily natural gas load forecasting based on the combination of long short term memory, local mean decomposition, and wavelet threshold denoising algorithm. J. Nat. Gas Sci. Eng. 95, 104175 (2021)
Pradhan, P., Nayak, B., Dhal, S.K.: Time series data prediction of natural gas consumption using ARIMA model. Int. J. Inf. Technol. Manag. Inf. Syst. 7(3), 1–7 (2016)
Sattler, F., Müller, K.R., Samek, W.: Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints. IEEE Trans. Neural Netw. Learn. Syst. 32(8), 3710–3722 (2020)
Sattler, F., Wiedemann, S., Müller, K.R., Samek, W.: Robust and communication-efficient federated learning from non-IID data. IEEE Trans. Neural Netw. Learn. Syst. 31(9), 3400–3413 (2019)
Taïk, A., Cherkaoui, S.: Electrical load forecasting using edge computing and federated learning. In: 2020 IEEE International Conference on Communications (ICC 2020), pp. 1–6. IEEE (2020)
Tan, A.Z., Yu, H., Cui, L., Yang, Q.: Toward personalized federated learning. In: IEEE Transactions on Neural Networks and Learning Systems (2022)
Wang, K., Mathews, R., Kiddon, C., Eichner, H., Beaufays, F., Ramage, D.: Federated evaluation of on-device personalization. arXiv preprint arXiv:1910.10252 (2019)
Wang, Y., Gao, N., Hug, G.: Personalized federated learning for individual consumer load forecasting. CSEE J. Power Energy Syst. (2022)
Wei, N., Li, C., Peng, X., Li, Y., Zeng, F.: Daily natural gas consumption forecasting via the application of a novel hybrid model. Appl. Energy 250, 358–368 (2019)
Xu, J., Wang, J., Long, M., et al.: Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv. Neural. Inf. Process. Syst. 34, 22419–22430 (2021)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. 10(2), 1–19 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, S., Liu, J., Ma, G., Yang, J., Liu, D., Li, Z. (2023). Two-Phased Federated Learning with Clustering and Personalization for Natural Gas Load Forecasting. In: Goebel, R., Yu, H., Faltings, B., Fan, L., Xiong, Z. (eds) Trustworthy Federated Learning. FL 2022. Lecture Notes in Computer Science(), vol 13448. Springer, Cham. https://doi.org/10.1007/978-3-031-28996-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-28996-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28995-8
Online ISBN: 978-3-031-28996-5
eBook Packages: Computer ScienceComputer Science (R0)