Abstract
Accurate long-term power forecasting is important in the decision-making operation of the power grid and power consumption management of customers to ensure the power system’s reliable power supply and the grid economy’s reliable operation. However, most time-series forecasting models do not perform well in dealing with long-time-series prediction tasks with a large amount of data. To address this challenge, we propose a parallel time-series prediction model called LDformer. First, we combine Informer with long short-term memory (LSTM) to obtain deep representation abilities in the time series. Then, we propose a parallel encoder module to improve the robustness of the model and combine convolutional layers with an attention mechanism to avoid value redundancy in the attention mechanism. Finally, we propose a probabilistic sparse (ProbSparse) self-attention mechanism combined with UniDrop to reduce the computational overhead and mitigate the risk of losing some key connections in the sequence. Experimental results on five datasets show that LDformer outperforms the state-of-the-art methods for most of the cases when handling the different long-time-series prediction tasks.
摘要
准确的长期电力预测对电网决策运行和用户用电管理非常重要, 可保证电力系统的可靠供电和电网经济的可靠运行。然而, 大多数时间序列预测模型在数据量大、预测精度高的长时间序列预测任务中表现不佳。为了应对这一挑战, 提出名为LDformer的并行时间序列预测模型。首先, 将Informer与长短期记忆网络相结合, 以获得时间序列的深度表达能力。其次, 提出并行编码器模块提高模型鲁棒性, 并将卷积层与注意力机制相结合, 以避免注意力机制中的值冗余。最后, 提出结合UniDrop的概率稀疏注意力机制, 以减少计算开销并减轻序列中一些关键连接丢失的风险。在5个真实数据集上的实验结果显示, 在不同的长时间序列预测任务中, LDformer大部分结果都优于最先进的基线结果。
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Cao JS, Wang JH, 2019. Stock price forecasting model based on modified convolution neural network and financial time series analysis. Int J Commun Syst, 32(12):e3987. https://doi.org/10.1002/dac.3987
Chakraborty T, Chattopadhyay S, Ghosh I, 2019. Forecasting dengue epidemics using a hybrid methodology. Phys A Stat Mech Appl, 527:121266. https://doi.org/10.1016/j.physa.2019.121266
Chen JF, Wang WM, Huang CM, 1995. Analysis of an adaptive time-series autoregressive moving-average (ARMA) model for short-term load forecasting. Electr Power Syst Res, 34(3):187–196. https://doi.org/10.1016/0378-7796(95)00977-1
Chen XY, Sun LJ, 2022. Bayesian temporal factorization for multidimensional time series prediction. IEEE Trans Patt Anal Mach Intell, 44(9):4659–4673. https://doi.org/10.1109/TPAMI.2021.3066551
Chuku C, Simpasa A, Oduor J, 2019. Intelligent forecasting of economic growth for developing economies. Int Econ, 159:74–93. https://doi.org/10.1016/j.inteco.2019.06.001
Ciechulski T, Osowski S, 2021. High precision LSTM model for short-time load forecasting in power systems. Energies, 14(11):2983. https://doi.org/10.3390/en14112983
Ding M, Zhou H, Xie H, et al., 2021. A time series model based on hybrid-kernel least-squares support vector machine for short-term wind power forecasting. ISA Trans, 108:58–68. https://doi.org/10.1016/j.isatra.2020.09.002
Gai SX, Zeng XQ, Yuan TF, 2021. Parking volume forecast of railway station garages based on passenger behaviour analysis using the LSTM network. J Adv Transp, 2021:6688609. https://doi.org/10.1155/2021/6688609
Guo SN, Lin YF, Li SJ, et al., 2019. Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting. IEEE Trans Intell Transp Syst, 20(10):3913–3926. https://doi.org/10.1109/TITS.2019.2906365
Han ZY, Zhao J, Leung H, et al., 2021. A review of deep learning models for time series prediction. IEEE Sens J, 21(6):7833–7848. https://doi.org/10.1109/JSEN.2019.2923982
Hosseini MK, Talebpour A, 2019. Traffic prediction using timespace diagram: a convolutional neural network approach. Transp Res Rec J Transp Res Board, 2673(7):425–435. https://doi.org/10.1177/0361198119841291
Hu R, Chiu YC, Hsieh CW, 2020. Crowding prediction on mass rapid transit systems using a weighted bidirectional recurrent neural network. IET Intell Transp Syst, 14(3):196–203. https://doi.org/10.1049/iet-its.2018.5542
Karevan Z, Suykens JAK, 2020. Transductive LSTM for time-series prediction: an application to weather forecasting. Neur Netw, 125:1–9. https://doi.org/10.1016/j.neunet.2019.12.030
Khan MH, Muhammad NS, El-Shafie A, 2020. Wavelet based hybrid ANN-ARIMA models for meteorological drought forecasting. J Hydrol, 590:125380. https://doi.org/10.1016/j.jhydrol.2020.125380
Khandelwal U, He H, Qi P, et al., 2018. Sharp nearby, fuzzy far away: how neural language models use context. Proc 56th Annual Meeting of the Association for Computational Linguistics, p.284–294. https://doi.org/10.18653/v1/P18-1027
Kim SH, Lee G, Kwon GY, et al., 2018. Deep learning based on multi-decomposition for short-term load forecasting. Energies, 11(12):3433. https://doi.org/10.3390/en11123433
Li SY, Jin XY, Xuan Y, et al., 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Proc 33rd Int Conf on Neural Information Processing Systems, Article 471. https://doi.org/10.5555/3454287.3454758
Marcjasz G, Uniejewski B, Weron R, 2020. Probabilistic electricity price forecasting with NARX networks: combine point or probabilistic forecasts? Int J Forecast, 36(2):466–479. https://doi.org/10.1016/j.ijforecast.2019.07.002
Miao KC, Han TT, Yao YQ, et al., 2020. Application of LSTM for short term fog forecasting based on meteorological elements. Neurocomputing, 408:285–291. https://doi.org/10.1016/j.neucom.2019.12.129
Min K, Kim D, Park J, et al., 2019. RNN-based path prediction of obstacle vehicles with deep ensemble. IEEE Trans Veh Technol, 68(10):10252–10256. https://doi.org/10.1109/TVT.2019.2933232
Nóbrega JP, Oliveira ALI, 2019. A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting. Neurocomputing, 337:235–250. https://doi.org/10.1016/j.neucom.2019.01.070
Ran XD, Shan ZG, Fang YF, et al., 2019. An LSTM-based method with attention mechanism for travel time prediction. Sensors, 19(4):861. https://doi.org/10.3390/s19040861
Syriopoulos T, Tsatsaronis M, Karamanos I, 2021. Support vector machine algorithms: an application to ship price forecasting. Comput Econ, 57(1):55–8. https://doi.org/10.1007/s10614-020-10032-2
Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. Proc 31st Int Conf on Neural Information Processing Systems, p.6000–6010. https://doi.org/10.5555/3295222.3295349
Viccione G, Guarnaccia C, Mancini S, et al., 2020. On the use of ARIMA models for short-term water tank levels forecasting. Water Supply, 20(3):787–799. https://doi.org/10.2166/ws.2019.190
Wang C, Tian R, Hu J, et al., 2023. A trend graph attention network for traffic prediction. Inform Sci, 623:275–292. https://doi.org/10.1016/j.ins.2022.12.048
Wang ZP, Qu JF, Fang XY, et al., 2020. Prediction of early stabilization time of electrolytic capacitor based on ARIMA-Bi_LSTM hybrid model. Neurocomputing, 403:63–79. https://doi.org/10.1016/j.neucom.2020.03.054
Wu N, Green B, Ben X, et al., 2020. Deep transformer models for time series forecasting: the influenza prevalence case. Proc 37th Int Conf on Machine Learning, p. 1–10.
Wu Z, Wu LJ, Meng Q, et al., 2021. UniDrop: a simple yet effective technique to improve transformer without extra cost. Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p.3865–3878. https://doi.org/10.18653/v1/2021.naacl-main.302
Xiao S, Yan JC, Farajtabar M, et al., 2019. Learning time series associated event sequences with recurrent point process networks. IEEE Trans Neur Netw Learn Syst, 30(10):3124–3136. https://doi.org/10.1109/TNNLS.2018.2889776
Xie YY, Lou YS, 2019. Hydrological time series prediction by ARIMA-SVR combined model based on wavelet transform. Proc 3rd Int Conf on Innovation in Artificial Intelligence, p.243–247. https://doi.org/10.1145/3319921.3319959
Xu DW, Wang YD, Jia LM, et al., 2017. Real-time road traffic state prediction based on ARIMA and Kalman filter. Front Inform Technol Electron Eng, 18(2):287–302. https://doi.org/10.1631/FITEE.1500381
Zhang D, Ling JW, Wei ZH, et al., 2018. Long-term traffic speed prediction based on multiscale spatio-temporal feature learning network. IEEE Trans Intell Transp Syst, 20(10):3700–3709. https://doi.org/10.1109/TITS.2018.2878068
Zhang L, Wang G, Giannakis GB, 2019. Real-time power system state estimation and forecasting via deep unrolled neural networks. IEEE Trans Signal Process, 67(15):4069–4077. https://doi.org/10.1109/TSP.2019.2926023
Zhang TJ, Song S, Li SG, et al., 2019. Research on gas concentration prediction models based on LSTM multidimensional time series. Energies, 12(1):161. https://doi.org/10.3390/en12010161
Zheng JH, Huang MF, 2020. Traffic flow forecast through time series analysis based on deep learning. IEEE Access, 8:82562–82570. https://doi.org/10.1109/ACCESS.2020.2990738
Zhou HY, Zhang SH, Peng JQ, et al., 2021. Informer: beyond efficient transformer for long sequence time-series forecasting. Proc 35th AAAI Conf on Artificial Intelligence, p.1–15. https://doi.org/10.1609/aaai.v35i12.17325
Author information
Authors and Affiliations
Contributions
Ran TIAN designed the research. Xinmei LI developed the methodology, curated the data, and worked on the software. Zhongyu MA conducted the investigation. Yanxing LIU processed the data. Jingxia WANG conducted the data visualization and result validation. Chu WANG verified the experimental results. Xinmei LI drafted the paper. Ran TIAN revised and finalized the paper.
Corresponding author
Ethics declarations
Ran TIAN, Xinmei LI, Zhongyu MA, Yanxing LIU, Jingxia WANG, and Chu WANG declare that they have no conflict of interest.
Additional information
Project supported by the National Natural Science Foundation of China (No.71961028), the Key Research and Development Program of Gansu Province, China (No. 22YF7GA171), the University Industry Support Program of Gansu Province, China (No. 2023QB-115), the Innovation Fund for Science and Technology-based Small and Medium Enterprises of Gansu Province, China (No. 23CXGA0136), and the Scientific Research Project of the Lanzhou Science and Technology Program, China (No.2018-01-58)
Rights and permissions
About this article
Cite this article
Tian, R., Li, X., Ma, Z. et al. LDformer: a parallel neural network model for long-term power forecasting. Front Inform Technol Electron Eng 24, 1287–1301 (2023). https://doi.org/10.1631/FITEE.2200540
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200540