1 Introduction

Energy production and consumption difference minimization is a challenging task these days. Efficient consumption of electricity is one good solution to this problem. Researchers has done a lot of work in for introducing cost effective and efficient energy utilization systems. Next generation Smart Grid (SG) is the most attractive solution so far. SG is the integration of information and communication technology in traditional grid which makes it intelligent power grid supporting real-time information exchange between producer and consumer. SG enables the energy efficiency optimization. More precisely, the SG needs an accurate forecasting of the energy load for more productive application.

AN increasing attempt of deregulating strength markets to shape a more dependable, green, and price-effective system with the aid of improving competitions has been witnessed in world’s important economies [1, 2]. In the liberalized markets, the strength is commoditized and consequently its price is dynamic. Due to the fee variant, pricing power correctly will become important to generate profits, schedule strength productions, and plan load responses [3,4,5,6,7]. The accurate power charge forecasting is helpful to decide the strength rate and accordingly is precious. As liberalized electricity markets include types, day-ahead and real-time [8], it’s miles meaningful to discuss both of the day-ahead and online forecasting of the power fee. The energy rate forecasting has been vigorously studied inside the literature. From the application aspect, the forecasting of the energy rate in unique deregulated markets of essential economies around the world has been stated [9,10,11,12,13,14,15].

The Rest of the paper is structured as defined next. The mostly used practicle techniques for load forecasting are discussed in Sect. 2. The Sect. 3 enlightens the different problems associated with the load prediction. The proposed model for load prediction and the evaluation metrices are explained in Sect. 4. Experimental results are depicted and highlighted in Sect. 5. Conclusion about the work done in this paper is expressed in Sect. 6 at the end.

2 Related Work

Authors in [16] proposed a novel practical methodology using quantile regression mean on a set of sisters point forecasts. Data from GEFCom2014 probabilistic load track was used to forecast for developing probabilistic load forecasts. The suggested scheme has dual advantages, where It will strength the advancement in the field of point load forecasting, it is not dependent on the high quality expert predictions. Scheme proposed in this work produces better results as compared to the benchmark methods. “Recency effect” a psychology term is used by authors in [17]. They exploited the fact that power consumption demand is influenced by the temperature of the earlier hours. Authors produced a ample study to show the effect of recency with the help of big data. Modern computing power is used in order to decide how many lagged temperature are required for catching the recency effect completely without affecting the predicting accuracy.

In [18] authors proposed a scheme for big data analytics in smart grid which aim at reducing the electricity cost for users. Moreover they explored the individual components needed for an improved decision support system for the purpose of energy saving. The presented framework has four different layers in its architecture, i.e., smart grid, data accumulation, an analysis counter and supporting web portal. Future power consumption is fore-casted and optimized through a innovative composite nature inspired meta heuristic prediction scheme. A versatile optimization algorithm works as a backbone for the analytics counter that helps in achieving accurate results. The proposed novel framework is the major contribution of this work, which supports the energy saving decision process. This contribution is the basis for full scale, Smart Decision Support System (SDSS). SDSS can identify the usage pattern of an individual user which helps in enhancing the efficiency of energy usage where improving the accuracy of fore-casted energy demands. Authors in [19] used forecasting analytics while focusing the extraction of related external features. More explicitly the proposed scheme predicts the spot prices in German energy market in relation to the historical data of prices and weather features. Least Absolute Shrinkage Selection Operation (LASSO) finds the related weather stations where implicit variable selection is executed by Random Forest (RF). This work enhanced the prediction accuracy with respect to Mean Average Error (MAE) by 16.9%.

A novel modeling scheme for electricity price prediction is introduced in [20]. Four different deep learning models are suggested by Lago et al. for forecasting electricity prices that lead to advancement in forecasting accuracy. Authors proposed despite the presence of a good number of electricity price forecasting methods, still a benchmark is missing. This work compared and evaluated 27 different common techniques used for electricity price prediction and then proved how the proposed models outperform the state of the art techniques those are significant. Wang et al. used Stacked De-noising Auto-encoder and Random Samples RS-SDA for live and next day hourly price prediction. In [21] short term forecasting of the electricity price is performed using data driven scheme. Deep Neural Networks type, SDA and its extended version RS-SDA are used to forecast the electricity price hourly for the data collected from different states of United States. This research is focused on next day hourly prediction and the live hourly prediction. SDA defined models are assessed in comparison with conventional neural network and support vector machine, where next day prediction SDA models accuracy is assessed in comparison industrial model.

In [22] Lagoa et al. introduced two distinct schemes for combining market incorporation in energy price prediction and to enhance the forecasting depiction. First scheme suggested a DNN that examines features from linked markets to enhance the forecasting results in a community market. Features importance is calculated using a innovative feature selection scheme that contains the optimization and functional analysis of variance. Second scheme forecasts the prices from two adjacent markets simultaneously which bring the accuracy metric Symmetric Mean Absolute Error (SMAPE) even further lower. Raviv et al. worked on predicting next day energy prices while utilizing hourly prices in [23]. This work exhibit that dismantled hourly rates include handy forecasting facts for the daily typical prices in the Nord pool market. It is evaluated that the multivariate patterns for the complete group of hourly prices considerably go better than univariate patterns of the daily normal price. Multivariate models reduce RMSE upto 16%. In [24] authors worked on electrical load forecasting on the basis of pre analysis and weight coefficients optimization. A novel scheme is introduced exploiting the features of electrical load data i.e., capacity to effectively calculate the seasonality and nonlinearity. The proposed new scheme can use up the advantages stay away from disadvantages of the individual schemes. In suggested combined scheme the data fore analyzation is adapted so that conflicts can be minimized in the data, where weight factors are adjusted using cuckoo search in the combined model. The newly proposed scheme outperforms the individual forecasting models regarding forecast performance.

Singh et al. worked on the amount of power consumed prediction in [25]. An intellectual data mining scheme is proposed that can evaluate, predict and reflect electricity time series to disclose numerous temporary energy using patterns. These patterns help to identify appliance usage relationship with time i.e., hour of day, week, month e.t.c, and appliance usage relationship with other appliances. This identification basis for the understanding the customer usage behavior, energy load forecasting and the price forecasting. Authors proposed Bayesian network forecasting, constant analysis of data through data mining and unsupervised data accumulating for electricity consumption prediction. In [26] authors proposed a short-lived electricity load prediction scheme for academic buildings. This work used 2-stage forecasting analysis for the productive working of their energy system. Energy consumption data is collected from different universities and moving average method is used for finding the energy load pattern according to week day. Random Forest (RF) technique is used for forecasting the daily energy load. RF performance is assessed using cross-validation on time series.

González et al. predicted electricity price adopting functional time series using a New Hilbertian ARMAX model in [27]. Suggested scheme has a linear regression structure, where functional variables are operated by functional parameters. Where functional parameters are fundamental entities with linearly combined kernels as sigmoid operations. Quasi-Newton model is used for parameters optimization in sigmoid which minimizes the sum of squared error. Data integrity attacks affect the results of load prediction models i.e., artificial neural network, multiple linear regression, support vector regression and fuzzy interaction regression). Authors in [28] worked on exposing the consequences of these attacks. We begin by simulating some knowledge integrity attacks through the random injection of some multipliers that follow a traditional or uniform distribution into the load series. Then, the four same load prognostication models are used to generate one-year-ahead ex post purpose forecasts so as to supply a comparison of their forecast errors. The results show that the support vector regression model is most robust, followed closely by the multiple rectilinear regression model, whereas the fuzzy interaction regression model is that the least sturdy of the four. withal, all four models fail to supply satisfying forecasts once the size of the info integrity attacks becomes giant. This presents a serious challenge to each load forecasters and therefore the broader prognostication community: the generation of correct forecasts beneath knowledge integrity attacks.

Dong et al. worked on the energy management in a microgrid. Bayesian-optimization-algorithm (BOA) is used for a single SG using house. Authors in [29] articulates the enhancement beyond the closed form equitable function equation, and work out on it using BOA based data-driven technique. We can consider the suggested technique as a black box function improving technique as a whole. Furthermore, it has the ability to handle the microgrid working and argument forecasting ambiguity.

3 Motivation and Problem Statement

Electricity load prediction is an important part of advanced power systems i.e., SG, effective power controlling, and improved energy operation engineering. Therefore, highly accurate prediction is needed for different perspectives, that are related to control, forwarding, planning and unit responsibility in a grid. Artificial Intelligence (AI) centered schemes has high competency to manipulate complicated mathematical problems, therefore, these techniques are widely employed in number of research areas.

The Artificial Neural Network (ANN) outperforms statistical schemes, as ANN is more efficient in mapping inputs to the outputs beyond complicated mathematical designs. Diverse learning structures are used by ANN for exploiting the linear association among the inputs [30]. ANN schemes has better depiction than analytical and time series techniques for prediction problems. The prediction performance in neural network is enhanced by the pre-processing of training data, high equivalence impact, optimal network structure and better learning algorithm. Moreover, ANN brings rapid confluence, minimized computing complexity, minimal training period and improved generalization [31].

Given a time series of 30 mins electricity loads, up to the time t, \(X_1\),.....\(X_t\), our goal is to predict load at time t+1, i.e., \({X_{t+1}}\).

4 Proposed System Model

Our proposed system forecasts the electricity load. In our proposed model we used dataset from Australian Energy Market Operator (AEMO). In this dataset electricity load recordings are taken after every 30 min, therefore producing 48 recordings for 48 time lags in a single day. Now if we want to predict the load of a time lag, we have just 48 features to use. To increase the number of features and better prediction we combine the records of week days from same week to form a single record yielding 336 recordings for a single record, i.e we have \(48\times 7\) features. The proposed system model is visualized in Fig. 1.

Now we calculate the importance of each feature in updated dataset using XGBoost feature selection technique. It help us to select the most appropriate features for selection. We consider 35–40 features with highest importance values for training and testing of our proposed scheme.

We have 1 year of data for 12 different regions, bringing \(12\times 365\) records for 365 days in a year. We divide the data into training and testing data as 75% training and 25% testing data. We train our proposed model using training data and perform testing using testing data. Algorithmic steps followed during the process of load prediction using XGBoost are defined in Algorithm 1.

figure a
Fig. 1.
figure 1

System model

Full size image

4.1 Evaluation Metrices

We used various standards for the evaluation of our proposed prediction model efficiency. The two most commonly used metrices for the measurement of prediction accuracy are Mean Absolute Percentage Error (MAPE) and Mean Absolute Error (MAE).

4.1.1 MAPE

The MAPE may be a live of prediction accuracy of a forecasting methodology for constructing fitted statistic values in statistics, specifically in trend estimation. it always expresses accuracy as a proportion of the error. as a result of this range may be a percentage, it may be easier to know than the opposite statistics. The MAPE is outlined as shown in (1). Here, and area unit the actual worth and therefore the forecast worth, severally. Also, is the number of times discovered.

$$\begin{aligned} MAPE=\frac{100}{n}\sum _{t=1}^{n} \left| \frac{A_t-F_t}{A_t} \right| \end{aligned}$$
(1)

4.1.2 MAE

In statistics, the MAE is employed to measure however shut forecasts or predictions area unit to the particular outcomes. It’s calculated by making a mean of absolutely the variations between the prediction values and therefore the actual ascertained values. The MAE is defined as shown in (2). Wherever nine is that the prediction price, is the actual price.

$$\begin{aligned} MAE=\frac{1}{n}\sum _{i=1}^{n} \left| {f_i-y_i} \right| =\frac{1}{n}\sum _{i=1}^{n} \left| e_i \right| \end{aligned}$$
(2)

5 Simulation Results and Discussion

We evaluated the performance of our proposed forecasting technique for predicting the load. We did perform a lot of experiments. The results obtained from extensive simulation are discussed here in this section.

5.1 Data Set Description

We used AEMO load data for the year 2017. The dataset records electricity load after every 30 mins making 48 lags daily. The considered dataset has records about 12 different areas for the same year 2017. We considered 365 days in a year in the provided dataset. Here Fig. 2 plots the load profile for individual week days. i.e., each weekday has its own line. From the graph we can see that Saturday has an overall highest load with respect to the other week days in the selected week. Furthermore, we can see that Wednesday has lowest load with respect to the other weekdays in the selected week. Where Fig. 3 is displaying the load of two consecutive weeks. It is clear from Figs. 2 and 3 that the electricity load data has daily as well as weekly cycles. The load at a specific time with respect to the other day is more or less same and it rises and falls more or less like the previous and next day. Similarly for the week, we can see that the load at one day in a week is more or less to the same day in previous and next week. These cycles are due to the cycles in daily human activities. For performing the load prediction of a specific time lag we combined the daily loads for same week to form a weekly dataset. i.e We combined the data from same weekdays to form single row. While conversion we neglected the weekdays that were not forming a complete week at the end of the year. This conversion provides more features for forecasting the load of a time lag.

Fig. 2.
figure 2

Daily load by weekdays

Full size image
Fig. 3.
figure 3

Two weeks load

Full size image

5.2 Prediction Model Configurations

We used XGBoost a gradient boosting framework, introduced back in 2014. XGBoost can be used as a forecasting technique for feature selection and load prediction of a time lag. From prediction to classification XGBoost has proved its worth in terms of performance.

When we convert the dataset to form a weekly data we have \(48\times 7\) recordings for a week. One row in the modified dataset represent a week with 336 features. To understand the importance of features, we used XGBoost to calculate the Feature Importance (FI) of all these features.

Figure 4 is depicting the FI of all these features. The greater the FI values means the feature will more effect the load prediction. It is evident from Fig. 4 that features with hight importance values are less in number, where most of the features have low importance value. We can see that the features close to the predicting lag have high importance as well. Also it is evident that days having same weekday for which we are predicting the load i.e., sunday have high importance value. We will only use the features with high FI values for the purpose of prediction and eliminate all other features from dataset. For selecting the features, we set a threshold for feature importance and we set this threshold by repeating experiment multiple times, and trying different threshold value. The value with best results in considered as threshold. The Fig. 4 shows that the features with high FI value are less in number, it will save the running time.

Fig. 4.
figure 4

XGB feature importance

Full size image

5.3 Forecasting Results

We used XGBoost for forecasting the load for a specific time lag in a week using weekly data. Figure 5 shows the real load in the dataset and the XGBoost fore-casted load. Here x-axis is depicting the time lags where y-axis is the load at that specific time lag. Actual load is represented by the blue graph, where fore casted load is represented by the green graph. We can see that the XGBoost load prediction follows the real load at most of the time, however at some high load instances XGBoost is not exactly following the real load. We can see that XGBoost is not predicting well at the high loads.

Fig. 5.
figure 5

XGB predictions

Full size image

The XGBoost load forecasting results for a time lag are displayed in Fig. 6. We can see that XGBoost forecasting technique results in a low Mean Average Percentage Error (MAPE), high accuracy and high Mean Average Error (MAE). XGBoost load prediction resulted in a 10.08% MAPE, 97.21% accuracy and 88.90% MAE.

Fig. 6.
figure 6

XGB results

Full size image

6 Conclusion

In this paper, we proposed a new scheme for electricity load forecasting. We converted daily electricity load information into weekly load information. It increases number of features available for predicting load for a lag variable. Then, we used XGBoost, a recently dominant machine learning technique for time series prediction, for feature selection from converted data. Once features are extracted we train the model using XGBoost. After training we use trained model for load prediction.