Abstract
Ensemble learning has gained considerable attention in different learning tasks including regression, classification, and clustering problems. One of the drawbacks of the ensemble is the high computational cost of training stages. Resampling local negative correlation (RLNC) is a technique that combines two well-known methods to generate ensemble diversity—resampling and error negative correlation—and a fine-grain parallel approach that allows us to achieve a satisfactory balance between accuracy and efficiency. In this paper, we introduce a structure of the virtual machine aimed to test diverse selection strategies of parameters in neural ensemble designs, such as RLNC. We assess the parallel performance of this approach on a virtual machine cluster based on the full virtualization paradigm, using speedup and efficiency as performance metrics, for different numbers of processors and training data sizes.
Similar content being viewed by others
References
Anastasiadis A, Magoulas G (2006) Analysing the localisation sites of proteins through neural networks ensembles. Neural Comput Appl 15:277–288. doi:10.1007/s00521-006-0029-y
Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In: SOSP ’03: proceedings of the nineteenth ACM symposium on operating systems principles. ACM, New York, pp 164–177
Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140
Breshears C (2009) The art of concurrency: a thread monkey’s guide to writing parallel applications. O’Reilly Media, California
Brown G (2004) Diversity in neural network ensembles. PhD thesis, School of Computer Science, University of Birmingham
Chai L, Gao Q, Panda DK (2007) Understanding the impact of multicore architecture in cluster computing: A case study with intel dual-core system. In: Cluster computing and the grid, 2007. CCGRID 2007. Seventh IEEE International Symposium on, pp 471–478
Crawford IL, Wadleigh KR (2000) Software optimization for high performance computers. Prentice Hall PTR, Upper Saddle River
Crosby S, Brown D (2007) The virtualization reality. Queue 4(10):34–41
Friedman JH (1991) Multivariate adaptive regression splines
Gepner P, Kowalik MF (2006) Multicore processors: new way to achieve high system performance. In: PARELEC ’06: proceedings of the international symposium on parallel computing in electrical engineering. IEEE Computer Society, Washington, DC, USA, pp 9–13
Grandvalet Y (2004) Bagging equalizes influence. Mach Learn 55(3):251–270
Gropp W, Lusk E, Skjellum A (1999) Using MPI: portable parallel programming with the message passing interface. MIT Press, Cambridge
Hiroaki I, Edahiro M, Sakai J (2007) Towards scalable and secure execution platform for embedded systems. In: ASP-DAC ’07: proceedings of the 2007 Asia and South pacific design automation conference. IEEE Computer Society, Washington, DC, USA, pp 350–354
Huang W, Liu J, Abali B, Panda DK (2006) A case for high performance computing with virtual machines. In: ICS ’06: proceedings of the 20th annual international conference on supercomputing. ACM, New York, NY, USA, pp 125–134
Liu B, Cui Q, Jiang T, Ma S (2004) A combinational feature selection and ensemble neural network method for classification of gene expression data. BMC Bioinformatics 5:136
Maqsood I, Khan MR, Abraham A (2004) An ensemble of neural networks for weather forecasting. Neural Comput Appl 13:112–122. doi:10.1007/s00521-004-0413-4
Ñanculef R, Valle C, Allende H, Moraga C (2006) Ensemble learning with local diversity. In: ICANN (1), pp 264–273
Ñanculef R, Valle C, Allende H, Moraga C (2006) Local negative correlation with resampling. In: IDEAL, pp 570–577
Necaise RD (2001) Using vmware for dual operating systems. J Comput Small Coll 17(2):294–300
Ong H, Farrell PA (2000) Performance comparison of lam/mpi, mpich, and mvich on a linux cluster connected by a gigabit ethernet network. In: ALS’00: proceedings of the 4th annual Linux Showcase & Conference. USENIX Association, Berkeley, CA, USA, pp 31–31
Ranadive A, Kesavan M, Gavrilovska A, Schwan K (2008) Performance implications of virtualizing multicore cluster machines. In: HPCVirt ’08: proceedings of the 2nd workshop on System-level virtualization for high performance computing. ACM, New York, NY, USA, pp 1–8
Valle C, Saravia F, Allende H, Monge R, Fernández C (2010) Parallel approach for ensemble learning with locally coupled neural networks. Neural Process Lett, 32:9157:277--9157:291
Watson J (2008) Virtualbox: bits and bytes masquerading as machines. Linux J 2008(166):1
Wells PM, Chakraborty K, Sohi GS (2009) Dynamic heterogeneity and the need for multicore virtualization. SIGOPS Oper Syst Rev 43(2):5–14
Whitaker A, Shaw M, Gribble SD (2002) Denali: lightweight virtual machines for distributed and networked applications. In: Proceedings of the USENIX Annual Technical Conference
Zhang PG (2007) A neural network ensemble method with jittered training data for time series forecasting. Inf Sci 177(23):5329–5346
Acknowledgments
This work was supported by the following Research Grants: Fondecyt 1070220, 1110854, and FB0821 Centro Cientìfico Tecnológico de Valparaíso. Partial support was also received from CONICYT (Chile) Ph.D. Grant 21080414, MECESUP Ph.D. Grant FSM0707-D3044, and project FB0821-FB_15LS_10.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fernández, C., Valle, C., Saravia, F. et al. Behavior analysis of neural network ensemble algorithm on a virtual machine cluster. Neural Comput & Applic 21, 535–542 (2012). https://doi.org/10.1007/s00521-011-0544-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-011-0544-3