Skip to main content
Log in

An improved approximation approach incorporating particle swarm optimization and a priori information into neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, an improved approach incorporating adaptive particle swarm optimization (APSO) and a priori information into feedforward neural networks for function approximation problem is proposed. It is well known that gradient-based learning algorithms such as backpropagation algorithm have good ability of local search, whereas PSO has good ability of global search. Therefore, in the improved approach, the APSO algorithm encoding the first-order derivative information of the approximated function is used to train network to near global minima. Then, with the connection weights produced by APSO, the network is trained with a modified gradient-based algorithm with magnified gradient function. The modified gradient-based algorithm can reduce input-to-output mapping sensitivity and lessen the chance of being trapped into local minima. By combining APSO with local search algorithm and considering a priori information, the improved approach has better approximation accuracy and convergence rate. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Nasr MB, Chtourou M (2009) A fuzzy neighborhood-based training algorithm for feedforward neural networks. Neural Comput Appl 18(2):127–133. doi:10.1007/s00521-007-0165-z

    Article  Google Scholar 

  2. Huang DS (2004) A constructive approach for finding arbitrary roots of polynomials by neural networks. IEEE Trans Neural Netw 15:477–491. doi:10.1109/TNN.2004.824424

    Article  Google Scholar 

  3. Huang DS, Chi ZR (2004) Finding roots of arbitrary high order polynomials based on neural network recursive partitioning method. Sci China Ser Inf Sci 47:232–245

    Article  MATH  MathSciNet  Google Scholar 

  4. Huang DS, Horace Ip HS, Chi ZR (2004) A neural root finder of polynomials based on root momnets. Neural Comput 16:1721–1762. doi:10.1162/089976604774201668

    Article  MATH  Google Scholar 

  5. Huang DS, Horace HS Ip, Chi ZR, Wong HS (2003) Dilation method for finding close roots of polynomials based on constrained learning neural networks. Phys Lett A 309:443–451. doi:10.1016/S0375-9601(03)00216-0

    Article  MATH  MathSciNet  Google Scholar 

  6. Han F, Huang DS (2008) A new constrained learning algorithm for function approximation by encoding a priori information into feedforward neural networks. Neural Comput Appl 17(5–6):433–439

    Google Scholar 

  7. Li SG, Wu ZM (2008) Business performance forecasting of convenience store based on enhanced fuzzy neural network. Neural Comput Appl 17(5–6):569–578

    Google Scholar 

  8. Jeong SY, Lee SY (2000) Adaptive learning algorithms to incorporate additional functional constraints into neural networks. Neurocomputing 35:73–90. doi:10.1016/S0925-2312(00)00296-4

    Article  MATH  Google Scholar 

  9. Jeong DG, Lee SY (1996) Merging back-propagation and Hebbian learning rules for robust classifications. Neural Netw 9:1213–1222. doi:10.1016/0893-6080(96)00042-1

    Article  Google Scholar 

  10. Han F, Huang DS, Cheung YM, Huang GB (2005) A new modified hybrid learning algorithm for feedforward neural networks, vol 3496. In: International symposium on neural network, Chongqing, 30 May–1 June, China. Lecture Notes in Computer Science, Springer, Berlin, pp 572–577

  11. Eberhart RC, Kennedy J (1995) A new optimizer using particles swarm theory. In: Proceeding of sixth international symposium on micro machine and human science, Nagoya, Japan, pp 39–43

  12. Eberhart RC, Kennedy J (1995) Particle swarm optimization, proceeding of IEEE International Conference on Neural Network, Perth, Australia, pp 1942–1948

  13. Parrott D, Li XD (2006) Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans Evol Comput 10(4):440–458. doi:10.1109/TEVC.2005.859468

    Article  Google Scholar 

  14. Blackwell T, Branke J (2006) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):459–472. doi:10.1109/TEVC.2005.857074

    Article  Google Scholar 

  15. Clerc M, Kennedy J (2002) The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73. doi:10.1109/4235.985692

    Article  Google Scholar 

  16. Parsopoulos K, Vrahatis M (2002) Recent approaches to global optimization problems through particle swarm optimization. Nat Comput 1(2–3):235–306. doi:10.1023/A:1016568309421

    Article  MATH  MathSciNet  Google Scholar 

  17. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading

    MATH  Google Scholar 

  18. Langdon WB, Poli R (2007) Evolving problems to learn about particle swarm optimizers and other search algorithms. IEEE Trans Evol Comput 11(5):561–578. doi:10.1109/TEVC.2006.886448

    Article  Google Scholar 

  19. Wang YP, Dang CY (2007) An evolutionary algorithm for global optimization based on level-set evolution and latin squares. IEEE Trans Evol Comput 11(5):579–595. doi:10.1109/TEVC.2006.886802

    Article  Google Scholar 

  20. Han F, Ling QH (2008) A new approach for function approximation incorporating adaptive particle swarm optimization and a priori information. Appl Math Comput 205(2):792–798. doi:10.1016/j.amc.2008.05.025

    Article  MATH  MathSciNet  Google Scholar 

  21. Ng SC, Cheung CC, Leung SH (2004) Magnified gradient function with deterministic weight modification in adaptive learning. IEEE Trans Neural Netw 15(6):1411–1423. doi:10.1109/TNN.2004.836237

    Article  Google Scholar 

  22. Shi YH, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of IEEE world conference on computation intelligence, pp 69–73

  23. Shi YH, Eberhart RC (1998) Parameter selection in particle swarm optimization. In: 1998 annual conference on evolutionary programming, San Diego, March

  24. Funahashi K (1989) On the approximate realization of continuous mapping by neural networks. Neural Netw 2(3):183–192. doi:10.1016/0893-6080(89)90003-8

    Article  Google Scholar 

  25. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal, approximators. Neural Netw 2(5):359–366. doi:10.1016/0893-6080(89)90020-8

    Article  Google Scholar 

  26. Irie B, Miyake S (1988) Capabilities of three-layered perceptions. In: Proceedings of the IEEE conference on neural networks, vol I. San Diego, CA, pp 641–648

Download references

Acknowledgments

This work was supported by the National Science Foundation of China (No. 60702056) and the Initial Funding of Science Research of Jiangsu University (No. 07JDG033).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fei Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, F., Ling, QH. & Huang, DS. An improved approximation approach incorporating particle swarm optimization and a priori information into neural networks. Neural Comput & Applic 19, 255–261 (2010). https://doi.org/10.1007/s00521-009-0274-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-009-0274-y

Keywords

Navigation