Abstract
In this paper, an improved approach incorporating adaptive particle swarm optimization (APSO) and a priori information into feedforward neural networks for function approximation problem is proposed. It is well known that gradient-based learning algorithms such as backpropagation algorithm have good ability of local search, whereas PSO has good ability of global search. Therefore, in the improved approach, the APSO algorithm encoding the first-order derivative information of the approximated function is used to train network to near global minima. Then, with the connection weights produced by APSO, the network is trained with a modified gradient-based algorithm with magnified gradient function. The modified gradient-based algorithm can reduce input-to-output mapping sensitivity and lessen the chance of being trapped into local minima. By combining APSO with local search algorithm and considering a priori information, the improved approach has better approximation accuracy and convergence rate. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach.
Similar content being viewed by others
References
Nasr MB, Chtourou M (2009) A fuzzy neighborhood-based training algorithm for feedforward neural networks. Neural Comput Appl 18(2):127–133. doi:10.1007/s00521-007-0165-z
Huang DS (2004) A constructive approach for finding arbitrary roots of polynomials by neural networks. IEEE Trans Neural Netw 15:477–491. doi:10.1109/TNN.2004.824424
Huang DS, Chi ZR (2004) Finding roots of arbitrary high order polynomials based on neural network recursive partitioning method. Sci China Ser Inf Sci 47:232–245
Huang DS, Horace Ip HS, Chi ZR (2004) A neural root finder of polynomials based on root momnets. Neural Comput 16:1721–1762. doi:10.1162/089976604774201668
Huang DS, Horace HS Ip, Chi ZR, Wong HS (2003) Dilation method for finding close roots of polynomials based on constrained learning neural networks. Phys Lett A 309:443–451. doi:10.1016/S0375-9601(03)00216-0
Han F, Huang DS (2008) A new constrained learning algorithm for function approximation by encoding a priori information into feedforward neural networks. Neural Comput Appl 17(5–6):433–439
Li SG, Wu ZM (2008) Business performance forecasting of convenience store based on enhanced fuzzy neural network. Neural Comput Appl 17(5–6):569–578
Jeong SY, Lee SY (2000) Adaptive learning algorithms to incorporate additional functional constraints into neural networks. Neurocomputing 35:73–90. doi:10.1016/S0925-2312(00)00296-4
Jeong DG, Lee SY (1996) Merging back-propagation and Hebbian learning rules for robust classifications. Neural Netw 9:1213–1222. doi:10.1016/0893-6080(96)00042-1
Han F, Huang DS, Cheung YM, Huang GB (2005) A new modified hybrid learning algorithm for feedforward neural networks, vol 3496. In: International symposium on neural network, Chongqing, 30 May–1 June, China. Lecture Notes in Computer Science, Springer, Berlin, pp 572–577
Eberhart RC, Kennedy J (1995) A new optimizer using particles swarm theory. In: Proceeding of sixth international symposium on micro machine and human science, Nagoya, Japan, pp 39–43
Eberhart RC, Kennedy J (1995) Particle swarm optimization, proceeding of IEEE International Conference on Neural Network, Perth, Australia, pp 1942–1948
Parrott D, Li XD (2006) Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans Evol Comput 10(4):440–458. doi:10.1109/TEVC.2005.859468
Blackwell T, Branke J (2006) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):459–472. doi:10.1109/TEVC.2005.857074
Clerc M, Kennedy J (2002) The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73. doi:10.1109/4235.985692
Parsopoulos K, Vrahatis M (2002) Recent approaches to global optimization problems through particle swarm optimization. Nat Comput 1(2–3):235–306. doi:10.1023/A:1016568309421
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading
Langdon WB, Poli R (2007) Evolving problems to learn about particle swarm optimizers and other search algorithms. IEEE Trans Evol Comput 11(5):561–578. doi:10.1109/TEVC.2006.886448
Wang YP, Dang CY (2007) An evolutionary algorithm for global optimization based on level-set evolution and latin squares. IEEE Trans Evol Comput 11(5):579–595. doi:10.1109/TEVC.2006.886802
Han F, Ling QH (2008) A new approach for function approximation incorporating adaptive particle swarm optimization and a priori information. Appl Math Comput 205(2):792–798. doi:10.1016/j.amc.2008.05.025
Ng SC, Cheung CC, Leung SH (2004) Magnified gradient function with deterministic weight modification in adaptive learning. IEEE Trans Neural Netw 15(6):1411–1423. doi:10.1109/TNN.2004.836237
Shi YH, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of IEEE world conference on computation intelligence, pp 69–73
Shi YH, Eberhart RC (1998) Parameter selection in particle swarm optimization. In: 1998 annual conference on evolutionary programming, San Diego, March
Funahashi K (1989) On the approximate realization of continuous mapping by neural networks. Neural Netw 2(3):183–192. doi:10.1016/0893-6080(89)90003-8
Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal, approximators. Neural Netw 2(5):359–366. doi:10.1016/0893-6080(89)90020-8
Irie B, Miyake S (1988) Capabilities of three-layered perceptions. In: Proceedings of the IEEE conference on neural networks, vol I. San Diego, CA, pp 641–648
Acknowledgments
This work was supported by the National Science Foundation of China (No. 60702056) and the Initial Funding of Science Research of Jiangsu University (No. 07JDG033).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Han, F., Ling, QH. & Huang, DS. An improved approximation approach incorporating particle swarm optimization and a priori information into neural networks. Neural Comput & Applic 19, 255–261 (2010). https://doi.org/10.1007/s00521-009-0274-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-009-0274-y