Abstract
This research investigates a swarm intelligence based multi-objective optimization algorithm for optimizing the behavior of a group of Artificial Neural Networks (ANNs), where each ANN specializes to solving a specific part of a task, such that the group as a whole achieves an effective solution. Niche Particle Swarm Optimization (NichePSO) is a speciation technique that has proven effective at locating multiple solutions in complex multivariate tasks. This research evaluates the efficacy of the NichePSO method for training a group of ANNs that form a neural network ensemble (NNE) for the purpose of solving a set of multivariate tasks. NichePSO is compared with a gradient descent method for training a set of individual ANNs to solve different parts of a multivariate task, and then combining the outputs of each ANN into a single solution. To date, there has been little research that has compared the effectiveness of applying NichePSO versus more traditional supervised learning methods for the training of neural network ensembles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blum, C., Li, X.: Swarm intelligence in optimization. In: Swarm Intelligence - Introduction and Applications, pp. 43–85. Springer, Heidelberg (2008)
Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, Oxford (1998)
Brits, R., Engelbrecht, A., van den Bergh, F.: A niching particle swarm optimizer. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning 2002 (SEAL 2002), pp. 692–696. MIT Press, Singapore (2002)
Brits, R., Engelbrecht, A., van den Bergh, F.: Locating multiple optima using particle swarm optimization. Applied Mathematics and Computation 189(1), 1859–1883 (2007)
Darwen, P., Yao, X.: Every niching method has its niche: fitness sharing and implicit sharing compared. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN IV 1996. LNCS, vol. 1141, pp. 398–407. Springer, Heidelberg (1996)
Engelbrecht, A., van den Bergh, F.: A new locally convergent particle swarm optimizer. In: Proceedings of the IEEE Conference on Systems, Man and Cybernetics, pp. 96–101. IEEE Press, Los Alamitos (2002)
Friedman, J., Grosse, E., Stuetzle, W.: Multidimensional additive spline approximation. Journal of Scientific and Statistical Computing 4(1), 291–301 (1983)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001
Heppner, F., Grenander, U.: A stochastic nonlinear model for coordinated bird flocks. In: Krasner, S. (ed.) The Ubiquity of Chaos, pp. 1–10. AAAS Publications, Washington (1990)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, pp. 231–238. MIT Press, Cambridge (1995)
Li, L., Zhou, J., Yu, X., Li, X.: Constrained power plants unit loading optimization using particle swarm optimization algorithm. WSEAS Transactions on Information Science and Applications 4(2), 296–302
Munro, P., Parmanto, B.: Competition among networks improves committee performance. In: Mozer, M., Petsche, J. (eds.) Advances in Neural Information Processing Systems, pp. 592–598. MIT Press, Cambridge (1997)
Reynolds, C.: Flocks, herds and schools: A distributed behavioral model. Computer Graphics 21(4), 25–36 (1987)
Rumelhart, D., Hinton, G., Williams, R.: Learning internal representations by error propagation: Parallel Distributed Processing. Foundations, vol. 1. MIT Press, Cambridge (1986)
Zhang, J., Zhang, J.-R., Li, K.: A sequential niching technique for particle swarm optimization. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 390–399. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Castillo, C., Nitschke, G., Engelbrecht, A. (2011). Niche Particle Swarm Optimization for Neural Network Ensembles. In: Kampis, G., Karsai, I., Szathmáry, E. (eds) Advances in Artificial Life. Darwin Meets von Neumann. ECAL 2009. Lecture Notes in Computer Science(), vol 5778. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21314-4_50
Download citation
DOI: https://doi.org/10.1007/978-3-642-21314-4_50
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21313-7
Online ISBN: 978-3-642-21314-4
eBook Packages: Computer ScienceComputer Science (R0)