Skip to main content
Log in

Behavior analysis of neural network ensemble algorithm on a virtual machine cluster

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Ensemble learning has gained considerable attention in different learning tasks including regression, classification, and clustering problems. One of the drawbacks of the ensemble is the high computational cost of training stages. Resampling local negative correlation (RLNC) is a technique that combines two well-known methods to generate ensemble diversity—resampling and error negative correlation—and a fine-grain parallel approach that allows us to achieve a satisfactory balance between accuracy and efficiency. In this paper, we introduce a structure of the virtual machine aimed to test diverse selection strategies of parameters in neural ensemble designs, such as RLNC. We assess the parallel performance of this approach on a virtual machine cluster based on the full virtualization paradigm, using speedup and efficiency as performance metrics, for different numbers of processors and training data sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Anastasiadis A, Magoulas G (2006) Analysing the localisation sites of proteins through neural networks ensembles. Neural Comput Appl 15:277–288. doi:10.1007/s00521-006-0029-y

    Article  Google Scholar 

  2. Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In: SOSP ’03: proceedings of the nineteenth ACM symposium on operating systems principles. ACM, New York, pp 164–177

  3. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  4. Breshears C (2009) The art of concurrency: a thread monkey’s guide to writing parallel applications. O’Reilly Media, California

    Google Scholar 

  5. Brown G (2004) Diversity in neural network ensembles. PhD thesis, School of Computer Science, University of Birmingham

  6. Chai L, Gao Q, Panda DK (2007) Understanding the impact of multicore architecture in cluster computing: A case study with intel dual-core system. In: Cluster computing and the grid, 2007. CCGRID 2007. Seventh IEEE International Symposium on, pp 471–478

  7. Crawford IL, Wadleigh KR (2000) Software optimization for high performance computers. Prentice Hall PTR, Upper Saddle River

    Google Scholar 

  8. Crosby S, Brown D (2007) The virtualization reality. Queue 4(10):34–41

    Article  Google Scholar 

  9. Friedman JH (1991) Multivariate adaptive regression splines

  10. Gepner P, Kowalik MF (2006) Multicore processors: new way to achieve high system performance. In: PARELEC ’06: proceedings of the international symposium on parallel computing in electrical engineering. IEEE Computer Society, Washington, DC, USA, pp 9–13

  11. Grandvalet Y (2004) Bagging equalizes influence. Mach Learn 55(3):251–270

    Article  MATH  Google Scholar 

  12. Gropp W, Lusk E, Skjellum A (1999) Using MPI: portable parallel programming with the message passing interface. MIT Press, Cambridge

    Google Scholar 

  13. Hiroaki I, Edahiro M, Sakai J (2007) Towards scalable and secure execution platform for embedded systems. In: ASP-DAC ’07: proceedings of the 2007 Asia and South pacific design automation conference. IEEE Computer Society, Washington, DC, USA, pp 350–354

  14. Huang W, Liu J, Abali B, Panda DK (2006) A case for high performance computing with virtual machines. In: ICS ’06: proceedings of the 20th annual international conference on supercomputing. ACM, New York, NY, USA, pp 125–134

  15. Liu B, Cui Q, Jiang T, Ma S (2004) A combinational feature selection and ensemble neural network method for classification of gene expression data. BMC Bioinformatics 5:136

    Article  Google Scholar 

  16. Maqsood I, Khan MR, Abraham A (2004) An ensemble of neural networks for weather forecasting. Neural Comput Appl 13:112–122. doi:10.1007/s00521-004-0413-4

    Google Scholar 

  17. Ñanculef R, Valle C, Allende H, Moraga C (2006) Ensemble learning with local diversity. In: ICANN (1), pp 264–273

  18. Ñanculef R, Valle C, Allende H, Moraga C (2006) Local negative correlation with resampling. In: IDEAL, pp 570–577

  19. Necaise RD (2001) Using vmware for dual operating systems. J Comput Small Coll 17(2):294–300

    Google Scholar 

  20. Ong H, Farrell PA (2000) Performance comparison of lam/mpi, mpich, and mvich on a linux cluster connected by a gigabit ethernet network. In: ALS’00: proceedings of the 4th annual Linux Showcase & Conference. USENIX Association, Berkeley, CA, USA, pp 31–31

  21. Ranadive A, Kesavan M, Gavrilovska A, Schwan K (2008) Performance implications of virtualizing multicore cluster machines. In: HPCVirt ’08: proceedings of the 2nd workshop on System-level virtualization for high performance computing. ACM, New York, NY, USA, pp 1–8

  22. Valle C, Saravia F, Allende H, Monge R, Fernández C (2010) Parallel approach for ensemble learning with locally coupled neural networks. Neural Process Lett, 32:9157:277--9157:291

    Google Scholar 

  23. Watson J (2008) Virtualbox: bits and bytes masquerading as machines. Linux J 2008(166):1

    Google Scholar 

  24. Wells PM, Chakraborty K, Sohi GS (2009) Dynamic heterogeneity and the need for multicore virtualization. SIGOPS Oper Syst Rev 43(2):5–14

    Article  Google Scholar 

  25. Whitaker A, Shaw M, Gribble SD (2002) Denali: lightweight virtual machines for distributed and networked applications. In: Proceedings of the USENIX Annual Technical Conference

  26. Zhang PG (2007) A neural network ensemble method with jittered training data for time series forecasting. Inf Sci 177(23):5329–5346

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the following Research Grants: Fondecyt 1070220, 1110854, and FB0821 Centro Cientìfico Tecnológico de Valparaíso. Partial support was also received from CONICYT (Chile) Ph.D. Grant 21080414, MECESUP Ph.D. Grant FSM0707-D3044, and project FB0821-FB_15LS_10.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Carlos Valle.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fernández, C., Valle, C., Saravia, F. et al. Behavior analysis of neural network ensemble algorithm on a virtual machine cluster. Neural Comput & Applic 21, 535–542 (2012). https://doi.org/10.1007/s00521-011-0544-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0544-3

Keywords

Navigation