Skip to main content
SpringerLink
Log in

Convolutional residual network to short-term load forecasting

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Since their inception, convolutional neural networks (CNNs) have been shown to have powerful feature extraction and learning capabilities, and the creation of deep residual networks (DRNs) was a milestone in the development of CNNs. However, residual networks mostly use convolution structures, which are widely applied to image recognition and classification problems. Therefore, when facing a load forecasting problem that involves nonlinear regression, will a DRN using a convolution structure still achieve great results? To answer this question, we present a network based on a DRN with a convolution structure to carry out short-term load forecasting, and we mainly focus on the effects of DRNs with different depths, widths and block structures for dealing with nonlinear regression problems. Through multiple sets of controlled experiments, we obtain the best network architecture and the corresponding hyperparameters for short-term load forecasting. The experimental results demonstrate that the model has higher prediction accuracy than existing models, and the DRN with a convolution structure can handle load forecasting while still achieving state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In this paper, based on different datasets, we consider different holidays. In the ISO-NE dataset, we consider Christmas Eve, Thanksgiving Day, and Independence Day as major holidays. In the Malaysia dataset, we consider Spring Festival, Eid, and Christmas Eve as major holidays. To simplify the model, the rest of the holidays are considered as nonholidays.

  2. Available at https://www.iso-ne.com/isoexpress/web/reports/load-and-demand

  3. Available at https://data.mendeley.com/datasets/f4fcrh4tn9/1

References

  1. Pang Q, Zhang M (2010) Very short-term load forecasting based on neural network and rough set. In: 2010 international conference on intelligent computation technology and automation, vol 3, IEEE, pp 1132–1135

  2. Chen K, Chen K, Wang Q, He Z, Hu J, He J (2018) Short-term load forecasting with deep residual networks. IEEE Transactions on Smart Grid

  3. Feilat E, Bouzguenda M (2011) Medium-term load forecasting using neural network approach. In: 2011 IEEE PES Conference on Innovative Smart Grid Technologies-Middle East, IEEE, pp 1–5

  4. Nalcaci G, Özmen A, Weber GW (2019) Long-term load forecasting: models based on mars, ann and lr methods. CEJOR 27(4):1033–1049

    Article  MathSciNet  Google Scholar 

  5. Li C, Li S, Liu Y (2016) A least squares support vector machine model optimized by moth-flame optimization algorithm for annual power load forecasting. Appl Intell 45(4):1166–1178

    Article  Google Scholar 

  6. Xu W, Peng H, Zeng X, Zhou F, Tian X, Peng X (2019) A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Applied Intelligence

  7. Hernández L, Baladrón C, Aguiar JM, Carro B, Sánchez-Esguevillas A, Lloret J (2014) Artificial neural networks for short-term load forecasting in microgrids environment. Energy 75:252–264

    Article  Google Scholar 

  8. Panapakidis IP (2016) Application of hybrid computational intelligence models in short-term bus load forecasting. Expert Systems with Applications 54:105–120

    Article  Google Scholar 

  9. Sengupta S, Basak S, Saikia P, Paul S, II RAP (2019) A review of deep learning with special emphasis on architectures, applications and recent trends

  10. Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377

    Article  Google Scholar 

  11. Di Persio L, Honchar O (2017) Analysis of recurrent neural networks for short-term energy load forecasting. In: AIP Conference Proceedings, vol 1906, AIP Publishing, pp 190006

  12. Staudemeyer RC, Morris ER (2019) Understanding lstm–a tutorial into long short-term memory recurrent neural networks. arXiv:1909.09586

  13. Zhao L, Zhou Y, Lu H, Fujita H (2018) Parallel computing method of deep belief networks and its application to traffic flow prediction. Knowledge Based Systems

  14. Kong W, Dong ZY, Jia Y, Hill DJ, Xu Y, Zhang Y (2017) Short-term residential load forecasting based on lstm recurrent neural network. IEEE Trans Smart Grid 10(1):841–851

    Article  Google Scholar 

  15. Duvenaud D, Rippel O, Adams R, Ghahramani Z (2014) Avoiding pathologies in very deep networks. In: Artificial intelligence and statistics, pp 202–210

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  17. Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv:1505.00853

  18. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580

  19. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  20. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211–252

    Article  MathSciNet  Google Scholar 

  21. Liu T, Chen M, Zhou M, Du SS, Zhou E, Zhao T (2019) Towards understanding the importance of shortcut connections in residual networks. In: Advances in neural information processing systems, pp 7890–7900

  22. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  23. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  24. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  25. Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv:1605.07146

  26. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500

  27. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  28. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, Springer, pp 630–645

  29. Lin M, Chen Q, Yan S (2013) Network in network. arXiv:1312.4400

  30. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

  31. Huang G, Li Y, Pleiss G, Liu Z, Hopcroft JE, Weinberger KQ (2017) Snapshot ensembles:, Train 1, get m for free. arXiv:1704.00109

  32. Sadaei HJ, Silva PCDLE, Guimaraes FG, Lee MH (2019) Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy 175:365–377

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of China under Grant 61803056, in part by China Postdoctoral Science Foundation under Grant 2017M620374, in part by the Natural Science Foundation of Chongqing cstc2018jcyjAX0365, and in part by Fundamental Research Funds for the Central Universities under Grant XDJK2018B013. The statements made herein are solely the responsibility of the authors.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Southwest University, Chongqing, 400715, People’s Republic of China

    Ziyu Sheng, Huiwei Wang & Jian Sun

  2. School of Automation, Central South University, Changsha, 410083, People’s Republic of China

    Guo Chen

  3. College of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing, 400074, People’s Republic of China

    Bo Zhou

Authors
  1. Ziyu Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huiwei Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sheng, Z., Wang, H., Chen, G. et al. Convolutional residual network to short-term load forecasting. Appl Intell 51, 2485–2499 (2021). https://doi.org/10.1007/s10489-020-01932-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01932-9

Keywords

Search

Navigation