Abstract
We examine properties for a batch training algorithm known as the output weight optimization–hidden weight optimization (OWO–HWO). Using the concept of equivalent networks, we analyze the effect of input transformation on BP. We introduce new theory of affine invariance and partial affine invariance for neural networks and prove this property for OWO–HWO. Finally, we relate HWO to BP and show that every iteration of HWO is equivalent to BP applied to whitening transformed data. Experimental results validate the connection between OWO–HWO and OWO–BP.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barton S (1991) A matrix method for optimizing a neural network. Neural Comput 3(3):450–459
Chen HH, Manry MT, Chandrasekaran H (1999) A neural network training algorithm utilizing multiple sets of linear equations. Neurocomputing 25(1–3):55–72. doi:10.1016/S0925-2312(98)00109-X
Dawson M, Fung A, Manry M (1993) Surface parameter retrieval using fast learning neural networks. Remote Sens Rev 7(1):1–18
De Bernardez L, Buitrago R, García N (2011) Photovoltaic generated energy and module optimum tilt angle from weather data. Int J Sustain Energy 30(5):311–320
Ebrahimpour R, Kabir E, Yousefi M (2007) Face detection using mixture of MLP experts. Neural Process Lett 26:69–82. doi:10.1007/s11063-007-9043-z
Ergezinger S, Thomsen E (1995) An accelerated learning algorithm for multilayer perceptrons: optimization layer by layer. IEEE Trans Neural Netw 6(1):31–42
Fahlman SE (1988) Faster-learning variations on back-propagation: an empirical study. In: Proceedings of the 1988 connectionist models summer school. Morgan Kaufmann, San Mateo, pp 38–51
Fletcher R (1987) Practical methods of optimization, 2nd edn. Wiley, New York
Güler E, Sankur B, Kahya Y, Raudys S (1998) Visual classification of medical data using MLP mapping. Comput Biol Med 28(3):275–287
Hagan M, Menhaj M (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5(6):989–993
Hornik K, Stinchcombe M, White H (1990) Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw 3(5):551–560
Kaminski W, Strumillo P (1997) Kernel orthonormalization in radial basis function neural networks. IEEE Trans Neural Netw 8(5):1177–1183
Kim T, Manry M, Maldonado J (2003) New learning factor and testing methods for conjugate gradient training algorithm. In: Proceedings of the international joint conference on neural networks, 2003. IEEE, vol 3, pp 2011–2016
LeCun Y, Bottou L, Orr G, Müller K (1998) Efficient backprop. In: Orr G, Müller KR (eds) Neural Networks: tricks of the trade, vol 1524. Lecture Notes in Computer Science. Springer, Berlin, pp 546–546
Li J, Manry M, Narasimha P, Yu C (2006) Feature selection using a piecewise linear network. IEEE Trans Neural Netw 17(5):1101–1115
Li J, Yao J, Petrick N, Summers R, Hara A (2006) Hybrid committee classifier for a computerized colonic polyp detection system. In: Proceedings of the SPIE medical imaging. Citeseer, San Diego
Manry MT (1982) Image processing and neural network laboratory. http://www-ee.uta.edu/eeweb/ip/new_training.html
Manry MT, Apollo SJ, Allen LS, Lyle WD, Gong W, Dawson M, Fung AK (1994) Fast training of neural networks for remote sensing. Remote Sens Rev 9:77–96. doi:10.1016/0893-6080(89)90020-8
Mantena V, Jiang W, Li J, McKenzie R (2009) Prostate cancer biomarker identification using maldi-ms data: initial results. In: Life science systems and applications workshop, 2009. LiSSA 2009. IEEE/NIH. IEEE, pp 116–119
Ondimu S, Murase H (2007) Reservoir level forecasting using neural networks: Lake Naivasha. Biosyst Eng 96(1):135–138
Parisi R, Di Claudio E, Orlandi G, Rao B (1996) A generalized learning paradigm exploiting the structure of feedforward neural networks. IEEE Trans Neural Netw 7(6):1450–1460
Raudys S (2001) Statistical and neural classifiers: an integrated approach to design. Springer, Berlin
Ruck D, Rogers S, Kabrisky M, Oxley M, Suter B (1990) The multilayer perceptron as an approximation to a Bayes optimal discriminant function. IEEE Trans Neural Netw 1(4):296–298. doi:10.1109/72.80266
Rumelhart D, Hinton G, Williams R (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536
Sartori M, Antsaklis P (1991) A simple method to derive bounds on the size and to train multilayer neural networks. IEEE Trans Neural Netw 2(4):467–471
Scalero R, Tepedelenlioglu N (1992) A fast new algorithm for training feedforward neural networks. IEEE Trans Signal Process 40(1):202–210
Sheikhan M, Shabani AA (2013) Pso-optimized modular neural network trained by OWO–HWO algorithm for fault location in analog circuits. Neural Comput Appl 23(2):519–530
Shepherd AJ (1997) Second-order methods for neural networks, 1st edn. Springer, New York
Wang G, Chen C (1996) A fast multilayer neural-network training algorithm based on the layer-by-layer optimizing procedures. IEEE Trans Neural Netw 7(3):768–775
Wang L (2009) Data mining with computational intelligence. Springer, Berlin
Yeh J (2006) Real analysis: theory of measure and integration. World Scientific Publishing Company Incorporated, Singapore
Yu C, Manry MT, Jiang L (2005) Effects of nonsingular preprocessing on feedforward network training. Int J Pattern Recognit Artif Intell 19(02):217–247. doi:10.1142/S0218001405004022
Yu Q, Apollo S, Manry M (1993) Map estimation and the multilayer perceptron. In: Proceedings of the 1993 IEEE-SP workshop on neural networks for processing [1993] III. IEEE, pp 30–39
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Robinson, M.D., Manry, M.T., Malalur, S.S. et al. Properties of a Batch Training Algorithm for Feedforward Networks. Neural Process Lett 45, 841–854 (2017). https://doi.org/10.1007/s11063-016-9553-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-016-9553-7