Skip to main content

Advertisement

SpringerLink
Log in

Sliding window-based LightGBM model for electric load forecasting using anomaly repair

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Smart grids have attracted much attention recently for their potential to reduce power system operating and management costs. Smart grid core components include energy storage, renewable energy source(s), and smart meters. Smart meters collect diverse data regarding smart grid operation, which can lead to inefficient operation if the meter data are damaged or tampered with during collection or transmission. Therefore, it is important to identify abnormalities in smart grid data and process them accordingly. Various anomaly detection models have been proposed using statistical methods, but they cannot detect some anomaly patterns accurately, and the models generally did not consider repair strategies for the detected anomalies. Anomaly repair should be included with model training to improve forecasting performance. This paper proposes a robust sliding window-based LightGBM model for short-term load forecasting using anomaly detection and repair. We first show how to detect anomalies using a variational autoencoder and then how they can be repaired using a random forest method. Finally, we verify that the proposed sliding window-based LightGBM achieves superior forecasting performance in combination with anomaly detection and repair.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hodge VJ, O’Keefe S, Weeks M, Moulds A (2014) Wireless sensor networks for condition monitoring in the railway industry: a survey. IEEE Trans Intell Transp Syst 16(3):1088–1106

    Article  Google Scholar 

  2. Hempstead M, Lyons MJ, Brooks D, Wei GY (2008) Survey of hardware systems for wireless sensor networks. J Low Power Electron 4(1):11–20

    Article  Google Scholar 

  3. Jagannathan S (2016) Real-time big data analytics architecture for remote sensing application. In: International Conference on Signal Processing, Communication, Power and Embedded System, pp 1912–1916 (2016).

  4. Kanoun O, Trankler HR (2004) Sensor technology advances and future trends. IEEE Trans Instrum Meas 53(6):1497–1501

    Article  Google Scholar 

  5. Bell WR, Hillmer SC (1983) Modeling time series with calendar variation. J Am Stat Assoc 78(383):526–534

    Article  Google Scholar 

  6. Fu TC (2011) A review on time series data mining. Eng Appl Artif Intell 24(1):164–181

    Article  Google Scholar 

  7. Ding D, Cooper RA, Pasquina PF, Fici-Pasquina L (2011) Sensor technology for smart homes. Maturitas 69(2):131–136

    Article  Google Scholar 

  8. Yu Z, Zheng X, Huang F, Guo W, Sun L, Yu Z (2020) A framework based on sparse representation model for time series prediction in smart city. Front Comp Sci 15(1):1–13

    Google Scholar 

  9. Chou JS, Ngo NT (2016) Smart grid data analytics framework for increasing energy savings in residential buildings. Autom Constr 72:247–257

    Article  Google Scholar 

  10. Tabrizchi H, Javidi MM, Amirzadeh V (2019) Estimates of residential building energy consumption using a multi-verse optimizer-based support vector machine with k-fold cross-validation. Evolv Syst 1–13 (2019)

  11. Park S, Moon J, Hwang E (2019) 2-Stage electric load forecasting scheme for day-ahead CCHP scheduling. In: IEEE 13th International Conference on Power Electronics and Drive Systems (PEDS), pp 1–4

  12. Montazerolghaem A, Moghaddam MHY, Leon-Garcia A (2017) OpenAMI: Software-defined AMI load balancing. IEEE Internet Things J 5(1):206–218

    Article  Google Scholar 

  13. Raciti M, Nadjm-Tehrani S (2013) Embedded cyber-physical anomaly detection in smart meters. In: Critical information infrastructures security, pp 34–45

  14. Jiang R, Lu R, Wang Y, Luo J, Shen C, Shen X (2014) Energy-theft detection issues for advanced metering infrastructure in smart grid. Tsinghua Sci Technol 19(2):105–120

    Article  Google Scholar 

  15. Moghaddass R, Wang J (2017) A hierarchical framework for smart grid anomaly detection using large-scale smart meter data. IEEE Trans Smart Grid 9(6):5820–5830

    Article  Google Scholar 

  16. Zhang W, Yang Q, Geng Y (2009) A survey of anomaly detection methods in networks. In: International symposium on computer network and multimedia technology, pp 1–3

  17. Wang C, Viswanathan K, Choudur L, Talwar V, Satterfield W, Schwan K (2011) Statistical techniques for online anomaly detection in data centers. In: 12th IFIP/IEEE international symposium on integrated network management and workshops, pp 385–392

  18. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):1–58

    Article  Google Scholar 

  19. Zhang A, Song S, Wang J, Yu PS (2017) Time series data cleaning: from anomaly detection to anomaly repairing. Proc VLDB Endowm 10(10):1046–1057

    Article  Google Scholar 

  20. Jung S, Moon J, Park S, Rho S, Baik SW, Hwang E (2020) Bagging ensemble of multilayer perceptrons for missing electricity consumption data imputation. Sensors 20(6):1772

    Article  Google Scholar 

  21. Armstrong JS (1989) Combining forecasts: the end of the beginning or the beginning of the end? Int J Forecast 5:585–588

    Article  Google Scholar 

  22. Breunig MM, Kriegel HP, Ng RT, Sander J (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp 93–104

  23. Liu FT, Ting KM, Zhou ZH (2008) Isolation forest. In: 8th IEEE International Conference on Data Mining, pp 413–422

  24. Chen J, Sathe S, Aggarwal C, Turaga D (2017) Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining, pp 90–98

  25. Akouemo HN, Povinelli RJ (2017) Data improving in time series using ARX and ANN models. IEEE Trans Power Syst 32(5):3352–3359

    Article  Google Scholar 

  26. Araya DB, Grolinger K, ElYamany HF, Capretz MA, Bitsuamlak G (2017) An ensemble learning framework for anomaly detection in building energy consumption. Energy Build 144:191–206

    Article  Google Scholar 

  27. Xu CD, Wang JF, Hu MG, Li QX (2013) Interpolation of missing temperature data at meteorological stations using P-BSHADE. J Clim 26(19):7452–7463

    Article  Google Scholar 

  28. Habermann C, Kindermann F (2007) Multidimensional spline interpolation: theory and applications. Comput Econ 30(2):153–169

    Article  Google Scholar 

  29. Gan S, Wang S, Chen Y, Zhang Y, Jin Z (2015) Dealiased seismic data interpolation using seislet transform with low-frequency constraint. IEEE Geosci Remote Sens Lett 12(10):2150–2154

    Article  Google Scholar 

  30. Jurado S, Peralta J, Nebot A, Mugica F, Cortez P (2013) Short-term electric load forecasting using computational intelligence methods. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1–8

  31. Grolinger K, L’Heureux A, Capretz MA, Seewald L (2016) Energy forecasting for event venues: Big data and prediction accuracy. Energy Build 112:222–233

    Article  Google Scholar 

  32. Abbasi RA, Javaid N, Ghuman MNJ, Khan ZA, Rehman SU (2019) Short term load forecasting using XGBoost. In: Workshops of the International Conference on Advanced Information Networking and Applications, pp 1120–1131

  33. Kuo PH, Huang CJ (2018) A high precision artificial neural networks model for short-term energy load forecasting. Energies 11(1):213

    Article  Google Scholar 

  34. Massana J, Pous C, Burgas L, Melendez J, Colomer J (2015) Short-term load forecasting in a non-residential building contrasting models and attributes. Energy Build 92:322–330

    Article  Google Scholar 

  35. Park S, Moon J, Jung S, Rho S, Baik SW, Hwang E (2020) A two-stage industrial load forecasting scheme for day-ahead combined cooling, heating and power scheduling. Energies 13(2):443

    Article  Google Scholar 

  36. Wang P, Liu B, Hong T (2016) Electric load forecasting with recency effect: a big data approach. Int J Forecast 32:585–597

    Article  Google Scholar 

  37. Xie J, Chen Y, Hong T, Laing TD (2016) Relative humidity for load forecasting models. IEEE Trans Smart Grid 9:191–198

    Article  Google Scholar 

  38. Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243

    Article  Google Scholar 

  39. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint http://arxiv.org/abs/1312.6114

  40. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Advances in neural information processing systems, pp 3146–3154

  41. Moon J, Jung S, Rew J, Rho S, Hwang E (2020) Combination of short term load forecasting models based on a stacking ensemble approach. Energy Build 109921

Download references

Acknowledgement

This research was supported by Energy Cloud R&D Program (grant number: 2019M3F2A1073184) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Korea University, Seoul, Republic of Korea

    Sungwoo Park, Seungmin Jung, Seungwon Jung & Eenjun Hwang

  2. Department of Industrial Security, Chung-Ang University, Seoul, Republic of Korea

    Seungmin Rho

Authors
  1. Sungwoo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungmin Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seungwon Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seungmin Rho
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eenjun Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eenjun Hwang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This paper is an extended version of our paper published in Proceedings of the 2020 International Conference on Artificial Intelligence (ICAI), Las Vegas, USA, 27–30 Jul 2020.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, S., Jung, S., Jung, S. et al. Sliding window-based LightGBM model for electric load forecasting using anomaly repair. J Supercomput 77, 12857–12878 (2021). https://doi.org/10.1007/s11227-021-03787-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03787-4

Keywords

Search

Navigation