Skip to main content
SpringerLink
Log in

Performance Evaluation of a New BP Algorithm for a Modified Artificial Neural Network

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In a conventional artificial neural model, the nonlinear activation function (AF) follows the weight sum operation. In this paper, the AF is placed before the connecting weights of each artificial neuron and hence modified artificial neural network (MANN) is proposed and the corresponding backpropagation (BP) learning algorithm is derived. Further, the slope of AF which is conventionally fixed during the training phase is adjusted for achieving better and faster training of the multi-layer artificial neural network (MANN). In this case, also both the weights and slope update algorithms are derived. To assess and compare the performance of these two ANN models, standard applications such as classification, nonlinear system identification (direct modeling), nonlinear channel equalization (inverse modeling) are implemented through simulation and compared with that obtained by conventional BP based MANN. The simulation results demonstrate that the adaptive slope based MANN outperforms the fixed slope as well as conventional MANN in terms of three different performance measures such as the number of iterations to converge, mean of the square of residual error and mean absolute deviation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back propagating errors. Nature 323:533–536

    MATH  Google Scholar 

  2. Jacobs RA (1988) Increased rates of convergence through learning rate adaptation. Neural Netw 1(4):295–307

    Google Scholar 

  3. Kollias S, Anastassiou D (1989) An adaptive least squares algorithm for the efficient training of artificial neural networks. IEEE Trans Circuits Syst 36(8):1092–1101

    Google Scholar 

  4. Widrow B, Lehr (1990) MA: 30 years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEE 78(9):1415–1442

    Google Scholar 

  5. Huang SC, Huang YF (1991) Bounds on the number of hidden neurons in multilayer perceptrons. IEEE Trans Neural Netw 2(1):47–55

    Google Scholar 

  6. Fukumi M, Omatu S (1991) A new backpropagation algorithm with coupled neuron. IEEE Trans Neural Netw 2(5):535–538

    Google Scholar 

  7. Charalambous C (1992) Conjugate gradient algorithm for efficient training of artificial neural networks. IEE Proc 139(3):301–310

    Google Scholar 

  8. Bernard E (1992) Optimization for training neural nets. IEEE Trans Neural Netw 3(2):232–240

    Google Scholar 

  9. Yam Y, Chow T (1993) Extended backpropagation algorithm. Electron Lett 29(19):1701–1702

    Google Scholar 

  10. Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5(6):989–993

    Google Scholar 

  11. Drago GP, Morando M, Ridella S (1995) An adaptive momentum back propagation (AMBP). Neural Comput Appl 3(4):213–221

    Google Scholar 

  12. Gori M, Maggini M (1996) Optimal convergence of on-line backpropagation. IEEE Trans Neural Netw 7:251–254

    Google Scholar 

  13. Tamura S, Tateishi M (1997) Capabilities of a four-layered feedforward neural network: four layers versus three. IEEE Trans Neural Netw 8(2):251–255

    Google Scholar 

  14. Verma B (1997) Fast training of multilayer perceptrons. IEEE Trans Neural Netw 8(6):1314–1320

    Google Scholar 

  15. Ng SC, Leung SH, Luk A (1999) Fast convergent generalized back propagation algorithm with constant learning rate. Neural Process Lett 9(1):13–23

    Google Scholar 

  16. Kamarathi SV, Pittner S (1999) Accelerating neural network training using weight extrapolations. Neural Netw 12(9):1285–1299

    Google Scholar 

  17. Patra JC, Pal RN, Chatterji BN, Panda G (1999) Identification of nonlinear dynamic systems using functional link artificial neural networks. IEEE Trans Syst Man Cybern Part B Cybern 29(2):254–262

    Google Scholar 

  18. Patra JC, Pal RN, Baliarsingh R, Panda G (1999) Nonlinear channel equalization for QAM signal constellation using artificial neural networks. IEEE Trans Syst Man Cybern Part B Cybern 29(2):254–262

    Google Scholar 

  19. Lera G, Pinzolas M (2002) Neighborhood based Levenberg–Marquardt algorithm for neural network training. IEEE Trans Neural Netw 13(5):1200–1203

    Google Scholar 

  20. Wang XG, Tang Z, Tamura H et al (2004) An improved backpropagation algorithm to avoid the local minima problem. Neurocomputing 56:455–460

    Google Scholar 

  21. Yang S-S, Ho C-L (2006) HBP: improvement in BP algorithm for an adaptive MLP decision feedback equalizer. IEEE Trans Circuits Syst II Exp Briefs 53(3):240–244

    MathSciNet  Google Scholar 

  22. Rimer M, Martinez T (2006) CB3: an adaptive error function for backpropagation training. Neural Process Lett 24(1):81–92

    Google Scholar 

  23. Siu S, yang SS, Lee CM, Ho CL (2007) Improving the back-propagation algorithm using evolutionary strategy. IEEE Trans Circuits Syst II 54(2):171–175

    Google Scholar 

  24. Wang CH, Kao CH, Lee WH (2007) A new iteractive model for improving the learning performance of back propagation neural network. Autom Constr 16:745–758

    Google Scholar 

  25. Haykin SS (2009) Neural networks and learning machines. Prentice Hall, Upper Saddle River

    Google Scholar 

  26. Burse K, Yadav RN, Shrivastava SC (2010) Channel equalization using neural networks: a review. IEEE Trans Syst Man Cybern Part C Appl Rev 40(3):352–357

    Google Scholar 

  27. Razavi S, Tolson BA (2011) A new formulation for feedforward neural networks. IEEE Trans Neural Netw 22(10):1588–1598

    Google Scholar 

  28. Hunter D, Yu H, Pukish MS, Kolbusz J, Wilamowski BM (2012) Selection of proper neural network sizes and architectures. IEEE Trans Ind Inform 8(2):228–240

    Google Scholar 

  29. Li LK, Shao S, Yiu KF (2013) A new optimization algorithm for single hidden layer feed forward neural network. Appl Soft Comput 13(1):2857–2862

    Google Scholar 

  30. Dai Q, Ma Z, Xie QY (2014) A two-phased and ensemble scheme integrated backpropagation algorithm. Appl Soft Comput 24:1124–1135

    Google Scholar 

  31. Wang L, Zeng Y, Chen T (2015) Backpropagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst Appl 42(2):855–863

    Google Scholar 

  32. Musikawan, Pakarat et al (2019) Parallelized metaheuristic-ensemble of heterogeneous feedforward neural networks for regression problems. IEEE Access 7:26909–26932

    Google Scholar 

  33. Smith JS, Wu B, Wilamowski BM (2018) Neural network training with Levenberg–Marquardt and adaptable weight compression. IEEE Trans Neural Netw Learn Syst 30:580–587

    Google Scholar 

  34. Patra JC, Meher PK, Chakraborty G (2011) Development of Laguerre neural-network-based intelligent sensors for wireless sensor networks. IEEE Trans Instrum Meas 60(3):725–734

    Google Scholar 

  35. Zhang Y, Chen B, Pan G, Zhao Y (2019) A novel hybrid model based on VMD–WT and PCA–BP–RBF neural network for short-term wind speed forecasting. Energy Convers Manag 195:180–197

    Google Scholar 

  36. Zhang Y, Zhao Y, Gao S (2019) A novel hybrid model for wind speed prediction based on VMD and neural network considering atmospheric uncertainties. IEEE Access 7(1):60322–60332

    Google Scholar 

  37. Zhang Y, Gao S, Ban M, Sun Y (2019) A method based on Lorenz disturbance and variational mode decomposition for wind speed prediction. Adv Electr Comput Eng 19(2):3–12

    Google Scholar 

  38. Zhang Y, Pan G, Zhang C, Zhao Y (2019) Wind speed prediction research with EMD-BP based on Lorenz disturbance. J Electr Eng 70(3):198–207

    Google Scholar 

  39. Li S, Chen T, Wang L, Ming C (2018) Effective tourist volume forecasting supported by PCA and improved BPNN using Baidu index. Tour Manag 68:16–126

    Google Scholar 

  40. Wang L, Wang Z, Hui Q, Liu S (2018) Optimal forecast combination based on neural networks for time series forecasting. Appl Soft Comput 66:1–17

    Google Scholar 

  41. Zeng Y-R, Zeng Y, Choi B, Wang L (2017) Multifactor-influenced energy consumption forecasting using enhanced back-propagation neural network. Energy 127:381–396

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. G.S. Sanayal School of Telecommunications, Indian Institute of Technology Kharagpur, Kharagpur, 721302, India

    Sashmita Panda

  2. Electronics and Communication Engineering, CV Raman Engineering College, Bhubaneswar, Odisha, India

    Ganapati Panda

Authors
  1. Sashmita Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ganapati Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sashmita Panda.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Panda, S., Panda, G. Performance Evaluation of a New BP Algorithm for a Modified Artificial Neural Network. Neural Process Lett 51, 1869–1889 (2020). https://doi.org/10.1007/s11063-019-10172-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10172-z

Keywords

Search

Navigation