Elsevier

Applied Soft Computing

Volume 52, March 2017, Pages 898-908
Applied Soft Computing

A pruning approach to optimize synaptic connections and select relevant input parameters for neural network modelling of solar radiation

https://doi.org/10.1016/j.asoc.2016.09.036Get rights and content

Highlights

  • An artificial neural network (ANN) model of solar radiations is subjected to a pruning process based on optimal brain surgeon (OBS).

  • Two main objectives are achieved, network optimization and selection of relevant input parameters.

  • A comparison is made with fully connected ANN.

  • The proposed approach is able to eliminate least significant parameters and at the same time optimizes the network structure to achieve a comparable performance.

Abstract

The accurate modelling of solar radiation is important for many applications including agriculture and energy management. Previous research has widely focused on the development of artificial neural network (ANN) models for this task. Since the inputs to these models include different meteorological parameters, it is usually time consuming and costly to acquire these parameters. Therefore, selection of most relevant input parameters is a key step in the construction of these models. The overall goal of this research is to develop ANN models with less number of parameters and at the same time, high modelling accuracy. To this aim, an algorithm for network pruning, the optimal brain surgeon (OBS), is proposed to achieve two main objectives, selection of most relevant input parameters and optimization of the network structure at synaptic level. Four meteorological parameters, namely: temperature (T), relative humidity (RH), wind speed (WS), and sunshine duration (SSD) are used to model solar radiation, the global horizontal irradiation (GHI), over Abu Dhabi, the United Arab Emirates. The results show that the least relevant input parameter is RH with a contribution of 15.2% in the modelling process as compared to 24.8% for WS, 47.8% for T, and 54.7% for SSD. The parameter selection results coincide with recently used techniques in solar radiation research, J48 technique in Waikato Environment for Knowledge Analysis (WEKA) software and Automatic Relevance Determination (ARD) method. The modelling performance is compared with an all-connected ANN using all the four input parameters and ten hidden layer neurons with a total of 61 synaptic connections including biases. On the one hand, the proposed technique has successfully achieved an ANN with three inputs (T, WS, SSD), seven active hidden layer neuron, and 17 synaptic connections including biases. On the other hand, there are improvements observed in statistical evaluation metrics including mean absolute biased error (MABE), adjusted coefficient of determination (R¯2), Akaike information criterion (AIC), Akaike final prediction error (FPE), Rissanen's minimum description length (MDL), and cross entropy (CE).

Introduction

There have been increasing concerns over global warming, greenhouse gas emissions and climate change for the last couple of decades. One of the major culprits in global warming is the coal and oil fired conventional electricity generation. This encourages the use of renewable energy sources as alternatives. These sources include wind, solar, hydro, wave, tidal, biomass, geothermal, and others. Due to its strategic position in the solar belt, the United Arab Emirates (UAE) has abundance of sunshine hours throughout the year with the yearly average global horizontal irradiance (GHI) of about 6 kWh/day [1]. This gives the UAE an opportunity to develop renewable energy technologies in the region and to promote clean and green environment. As solar radiation is measured with different ground based sensors including pyranometers, pyroheliometers, and radiometers, etc. combined with data acquisition hardware and software, it is time consuming, cumbersome, and expensive to install these sensors everywhere. This gives more importance to solar radiation models. The accuracy of the solar radiation models directly relates to the accuracy of modelling power output from solar plants and that consequently affects the management and planning of future renewable energy systems.

There are many soft computing approaches to solar radiation modelling that include genetic programming (GP) [2], particle swarm optimization (PSO) [3], hybrid coral-reef algorithms [4], neuro-fuzzy [5], fire-fly [6] and more. Many researchers have used the artificial neural network (ANN) techniques to model solar energy. Multi-layer perceptron (MLP), with one hidden layer, is the most widely used ANN for solar radiation applications [7], [8], [9], [10], [11]. Recently, a very comprehensive review on the use of ANN for time series modelling in solar radiation applications is presented by Yadev & Chandel [12]. The authors have pointed out a need to identify the most relevant parameters to be used for ANN models and omit the non-relevant ones. One of the reasons is the capital costs of the required sensors that are directly related with the number of parameters measured. Another reason is the expensive data acquisition systems and the amount of hardware and software needed to process the data.

Different authors have addressed this problem in different ways. Yadav et al. [13] found that the modelling accuracy of the ANN varied with change in number of input parameters. They used Waikato Environment for Knowledge Analysis (WEKA) software for selection of most relevant input parameters to be used for solar radiation applications in different states of India. WEKA is a collection of machine learning programs together offering one single user interface. WEKA uses J48 algorithm (an implementation of c4.5 [14] algorithm in WEKA) to construct decision trees for classification. WEKA determines a rank for each input variable and ones with the lowest rank are discarded. The authors studied the selection of relevant input parameters from a set of candidates that included, T, minimum temperature (Tmin), maximum temperature (Tmax), altitude (Alt), SSD, latitude (Lat), and longitude (Lon). The authors found that the geographical parameters, Lat and Lon had the lowest rank for each month. The ranking of the parameters as determined by WEKA was T, Tmin, Tmax, Alt, SSD, Lat, and Lon from high to low priority. The authors simulated three different structures of ANN. ANN-1 with 7 inputs (T, Tmin, Tmax, Alt, SSD, Lat, and Lon), ANN-2 with 5 inputs (T, Tmin, Tmax, Alt, and SSD) and ANN-3 with 4 inputs (T, Tmin, Tmax, and Alt). The output layer dimension for all the ANN architectures was 1. The authors also varied the number of hidden layers from 9 to 19 for ANN-1, from 8 to 18 for ANN-2 and from 7 to 18 for ANN-3. The authors found out that ANN-1 with the structure of 7-9-1 gave better performance with mean absolute percent error (MAPE) of 20.12%. The ANN-2 with the structure of 5-10-1 gave better performance with MAPE of 6.89% and the ANN-3 with the structure of 4-10-1 gave better performance with MAPE of 9.09%. It was therefore concluded that ANN-2 with Lat, Lon excluded gave better performance and ANN-1 with Lat, Lon included degraded the modelling performance. An interesting finding was the exclusion of SSD that degraded the performance of ANN-3 as compared to ANN-2 with SSD included. A second study was performed by the same authors using WEKA [15] for the same dataset. Two additional variables, extraterrestrial radiations (ER) and clearness index (CI) were added. The authors concluded lower ranks to clearness index, extraterrestrial radiation, Lat, and Lon. Five different ANN architectures were simulated and it was found that the ANN with the same input parameters as mentioned previously (ANN-2) gave better performance.

López et al. [16] used automatic relevance determination (ARD) method to select relevant input parameters in modelling direct normal solar irradiance for Desert Rock (USA). The ARD is a Bayesian approach for the ANN [17], [18], [19] that considers a priori probability distribution over the network weights and determines the most relevant input parameters by introducing a hyperparameter for each input unit of the ANN. The prior distribution of the network weights are controlled by the hyerparameters. The ANN is presented with the training data and the posterior distribution of weights and hyperparameters are calculated using Bayes rule. After the training is finished, the relevancy of the inputs is determined by the value of hyperparameter associated with that particular input. A comparatively large value of hyperparameter indicates less importance of the corresponding parameter. The authors used T, RH, pressure (p), WS, dew-point temperature (Td), CI, precipitable water (w), and relative optical air mass (mr) as candidate parameters for ANN with 2 and 10 hidden layer nodes. The ARD method found that the CI and mr were the most relevant parameters to be used in their case. Bosch et al. [20] also used ARD and reported the estimation errors for different combinations of the input parameters over a mountainous area in Spain. The input parameters used were Lat, Lon, Alt, slope of ground station, azimuth, day of year (DOY), daily extraterrestrial solar radiation, and daily CI. The ANN models daily global irradiation. The authors found that Alt was a most relevant geographical parameter along with the DOY and daily CI. After the determination of important parameters, the authors used different ANN architectures with varying number of hidden units from 3 to 30. They trained every architecture 10 times to cater for random nature of weights initialization and to increase reliability of the results. The authors observed that the ANN with 14 hidden neurons gives better performance in this case and remains almost constant for higher number of hidden units. For the final ANN, the value of root-mean-squared error (RMSE) was recorded as 6.0% and mean biased error (MBE) as 0.2%. Yacef et al. [21] also used ARD for selecting an optimum combination of input parameters for ANN to estimate solar radiation in Al-Madinah, Saudi Arabia. The authors used T, RH, SSD and extraterrestrial irradiation (EI) as input parameters. The ARD determined EI and RH as comparatively less relevant parameters because of high values of hyperparameters associated with these variables. Therefore, the authors used T and SSD as two inputs to different ANN architectures with hidden neurons varying from 1 to 30 and found that the ANN with one hidden layer and two hidden neurons was the best architecture with RMSE of 8.4137%, MBE of 3.0658% and mean absolute error (MAE) of 5.9092%. Will et al. [22] used a niching genetic algorithm (NGA) for selection of input parameters to estimate solar radiations in El Colmenar (Tucuman, Argentina). The NGA is a modified form of the normal GA where all realizations of global and local optimums are found and for a same estimation error, the algorithm produces several combinations of input parameters. The NGA performs categorization of the problem, assigns limits to each input parameter, and calculates the distance in the search space. The authors used data from 14 different stations in Argentina and found out that T, RH, p, and SSD were the relevant parameters. The values of RMSE were reported to be 2.36 MJ/m2 for 70 individuals/85 generations and 2.34 MJ/m2 for 200 individuals/150 generations. Many authors on the other hand, used trial and error methods to determine the most relevant set of input parameters used for ANN to estimate solar radiations. Alam et al. [23] developed sixteen different ANN models to estimate solar radiation for different Indian stations. Each model used a different set of input parameters. A total of ten input parameters including Lat, Lon, Alt, time (t), the month of the year (MOY), T, RH, rainfall (RF), WS, and a net long wavelength (LW) were considered to develop these sixteen models. They also used ANN models with a varying number of hidden layers and trained for different seasons of the year. They found Lat, Lon, Alt, t, MOY, AT, RH, WS, and LW parameters were suitable for the summer season with ANN architecture containing 13 hidden neurons and coefficient of determination (R2) value of 0.931. For the raining season, the best parametric combination was Lat, Lon, Alt, t, MOY, AT, RH, RF, WS, LW and ANN architecture with 11 hidden neurons and R2 value of 0.907. For the winter season, the best parametric combination was Lat, Lon, Alt, t, MOY, AT, RH, LW and ANN architecture with 9 hidden neurons and R2 value of 0.923.

Ahmet et al. [24] reported the effects of the number of input parameters on the estimation error. They changed the input layer parameters for ANN from 2 to 6 and recorded the relevant statistical error indicators for five different cities in Turkey. They concluded that training the ANN with all the six parameters (Lat, Lon, Alt, month, average cloudiness, and SSD) as inputs and 8 hidden neurons gave better performance with average value of correlation coefficient (R) equal to 0.9896 and best value of RMSE as 3.58%.

Hassan et al. [25] studied eleven possible combinations of four input parameters including T, WS, SSD, and RH for different cities of the UAE. The authors varied the number of hidden neurons in the range of 5–500 and tried different combinations of the input parameters. They reported the network structures and statistical errors for optimal ANN models for three cities of the UAE. For Abu Dhai, UAE, the optimal network structure had 60 hidden neurons with three input parameters (T, SSD, RH), RMSE of 0.304 kWh/m2 and R2 of 94%. For Al Ain, UAE, the optimal network structure had 100 hidden neurons with all four inputs, RMSE of 0.349 kWh/m2 and R2 of 92%. For Dubai, UAE, the optimal network structure had 205 hidden neurons with three input parameters (T, SSD, WS), RMSE of 0.318 kWh/m2 and R2 of 95%.

Behrang et al. [26] used six different combinations of input parameters including the day of the year (DOY), T, RH, SSD, evaporation, and WS to estimate global solar radiation in Iran, using ANN. Authors also used a varying number of hidden layer neurons and reported the best performance for the parametric set of DOY, T, RH, SSD, and WS combined with ANN consisting of two hidden layers of 3 neurons each. The values of MSE and R2 were 0.00395 and 99.57%, respectively.

Tymvios et al. [27] presented seven different ANN models with different combinations of input parameters, hidden layers, and the number of hidden neurons to model solar radiation over Cyprus. They showed that theoretical daily SSD has more relevance as compared to the month serial number. They also concluded that increasing the number of hidden layers did not necessarily improve the performance. Rather, ANN with more hidden layers tended to remember all the training patterns and this degraded the generalization capabilities of the ANN model because of over fitting. The authors reported the architecture of the best ANN model with three inputs (T, measured SSD and theoretical SSD) and two hidden layers with 46 and 23 neurons in each layer. The values of MBE and RMSE were reported as 0.12% and 5.67%, respectively.

Fadare [28] used different combinations of hidden neurons to model solar energy potential for 195 cities in Nigeria for the period of 10 years. He used seven different input parameters (Lat, Lon, Alt, month, mean SSD, mean T, and RH) to construct different ANN models with one or two hidden layers having a varying number of neurons. He concluded that ANN with one hidden layer and five hidden neurons gave the best performance with R values of 97.8%, 97.1%, and 95.6% for training, testing, and the whole dataset, respectively. Table 1 lists a summary of the literature review in parameter relevance selection for solar radiation modelling using ANN based methods.

From the literature review, two different problems are identified, selecting the relevant input parameters and optimizing the network structure of the ANN. There are many techniques in the available literature that address these problems. These techniques either search for optimum number of parameters or optimize the network structure at node level. In [29] and [30], authors used PSO to select optimum parameters for ANN in forecasting wind speed and rainfall. Another partial mutual information-based input variable selection method tested on synthetic and storm water data is reported in [31] and [32].

However, there is a limitation in using systematic methods in solar radiation literature and there have been a very few systematic attempts like WEKA, ARD, and NGA. These techniques are able to select relevant input parameters and consequently, optimize the all-connected ANN, but do not optimize the network structure at synaptic level. Too small network may not learn the problem well because of under-fitting and too large network may not generalize the problem because of over-fitting. The novelty of this study is to use a systematic approach to solve both the problems, relevancy of the input parameters and synaptic level optimization of the network, simultaneously.

This study proposes the use of optimal brain surgeon (OBS) strategy [33] integrated with single hidden layer MLP for network pruning in solar radiation research. The OBS is used in many other applications. A use of OBS for ANN is demonstrated on Monk's problem in [34]. Many other usages of OBS are demonstrated in [35] and [36]. To the best of author's knowledge, no application of OBS in solar radiation research is found. The proposed technique not only selects relevant input parameters but also optimizes the ANN structure at synaptic level. The rest of the paper is organized as follows. Section 2 discusses the basics of MLP and OBS algorithms along with the integration strategy. Data collected and the removal of multivariate outliers from the data based on squared Mahalanobis distance are discussed in Section 3 and simulations in Section 4. Finally, Section 5 concludes the paper.

Section snippets

The methodology

This section discusses the basic theory behind both the techniques and how the integration of MLP and OBS is achieved to solve both the above mentioned problems, namely: selecting relevant input parameters and optimizing the ANN structure.

Data collection and quality control

The selected data are from Abu Dhabi International Airport, UAE at geographical position (24.44 °N, 54.65 °E). The data spans over the range of ten years (2004–2013) and contains daily averages of air temperature (T: °C), relative humidity (RH: %), wind speed (WS: knots), sunshine duration (SSD: hours), and global horizontal irradiation (GHI: Wh/m2). Fig. 3 shows the time evolution of these meteorological variables. In this section, the main purpose of data quality control is to identify any

Simulations and discussions

As mentioned above, a relatively large MLP is selected and trained through backpropagation algorithm to minimum error. In this section, an MLP is initialized with one input layer, one hidden layer, and one output layer. There are four input units (n = 4), twenty hidden units (m = 20), and one output unit (p = 1). This makes the total number of parameters equal to 121 including biases. ΦH is chosen as a sigmoid activation functions and Φo is chosen as a linear activation functions. The inputs to the

Conclusions

The scarcity of an automated method for selection of relevant input parameters and network structure optimization in solar radiation applications is evident. It has been found that except a very few automated approaches, trial and error methods are used by the research community in this field. Moreover, the automated methods in solar radiation applications are limited to selection of relevant input parameters and optimization of all-connected neural networks only. The purpose of integrating

References (43)

  • K. Moustris et al.

    Use of neural networks for the creation of hourly global and diffuse solar irradiance data at representative locations in Greece

    Renew. Energy

    (2008)
  • A.K. Yadav et al.

    Solar radiation prediction using artificial neural network techniques: a review

    Renew. Sustain. Energy Rev.

    (2014)
  • A.K. Yadav et al.

    Selection of most relevant input parameters using WEKA for artificial neural network based solar radiation prediction models

    Renew. Sustain. Energy Rev.

    (2014)
  • A.K. Yadav et al.

    Solar energy potential assessment of western Himalayan Indian state of Himachal Pradesh using J48 algorithm of WEKA in ANN based prediction model

    Renew. Energy

    (2015)
  • G. López et al.

    Selection of input parameters to model direct solar irradiance by using artificial neural networks

    Energy

    (2005)
  • J. Lampinen et al.

    Bayesian approach for neural networks – review and case studies

    Neural Netw.

    (2001)
  • W.D. Penny et al.

    Bayesian neural networks for classification: how useful is the evidence framework?

    Neural Netw.

    (1999)
  • J.L. Bosch et al.

    Daily solar irradiation estimation over a mountainous area using artificial neural networks

    Renew. Energy

    (2008)
  • R. Yacef et al.

    Prediction of daily global solar irradiation data using Bayesian neural network: a comparative study

    Renew. Energy

    (2012)
  • A. Will et al.

    On the use of niching genetic algorithms for variable selection in solar radiation estimation

    Renew. Energy

    (2013)
  • S. Alam et al.

    Assessment of diffuse solar energy under general sky condition using artificial neural network

    Appl. Energy

    (2009)
  • Cited by (26)

    • The role of input selection and climate pre-classification on the performance of neural networks irradiance models

      2022, Applied Soft Computing
      Citation Excerpt :

      The suitability of the inputs used to design the ANN usually relies on the designer criteria or physical factors affecting solar irradiance. Nowadays, there have been a few, but valuable efforts focused on understanding the effect of the input variables on the accuracy of the ANN models and optimizing their utilization as solar irradiance estimators [5,30,31]. However, none of these works considers the climate classification as an input and its effect on the ANN.

    • Global solar radiation prediction over North Dakota using air temperature: Development of novel hybrid intelligence model

      2021, Energy Reports
      Citation Excerpt :

      Measuring all the properties of solar radiation requires relatively expensive sensors like radiometers, pyranometers, and pyrheliometers incorporated with data-acquisition software and hardware (Dong and Jiang, 2019). The installation of such equipment and sensors across the world are also time-consuming and cumbersome (Hussain and Alili, 2017). To address these obstacles, it is very necessary to develop reliable SR prediction models with easily available meteorological variables for accurate estimation of SR at any point of interest.

    • Adaptive co-optimization of artificial neural networks using evolutionary algorithm for global radiation forecasting

      2021, Renewable Energy
      Citation Excerpt :

      Di Piazza et al. [49] developed a nonlinear autoregressive network with exogenous input model optimized by genetic algorithm and optimal brain surgeon strategy to forecast hourly solar irradiation and wind speed in Sicily, Italy. Hussain and Al Alili [50] developed ANN models with less number of parameters and high modeling accuracy. The optimal brain surgeon was proposed to achieve these objectives using the selection of the most relevant input parameters and the optimization of the network structure at the synaptic level.

    • Facial expression recognition with polynomial Legendre and partial connection MLP

      2021, Neurocomputing
      Citation Excerpt :

      On the other hand, in terms of the noise added to the weights in two deep networks with the same learning algorithm, the network with lower number of weights performs better than the network with higher number of weights [32]. Also, according to [33], in MLP network structure, partial synaptic connections can be used to produce more efficient networks. A network can be modified in a way that the additional connections between different nodes are removed without any change in performance.

    View all citing articles on Scopus
    View full text