Skip to main content
Log in

A self-organized volunteer Cloud for e-Science

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Nowadays, the adoption of Cloud Computing platforms and Service Computing technologies are almost natural for the different e-Science communities. Cost benefits for data-intensive applications, ease of access, rich and varied offers for services are examples of positive returns by users. However, beyond this favorable welcome for the technology, some research problems remain and are still challenging. In this paper, we focus on the problems of automatically deploying IaaS for computing and for data management, using the SlapOS Cloud. The core of the system is a distributed protocol for orchestrating data and compute nodes. Using this interaction scheme, users are able to deploy, without any system administrator intervention, a PaaS inside the IaaS basically a Desktop Grid middleware. The aim of this paper is to demonstrate that the Desktop Grid and Cloud paradigms may merge and may be widely used by non-experts in the different areas of e-Science. We propose a fully self-organized volunteer Cloud for researchers where they can carry out e-Science experiments and process large amounts of data in a coherent way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.xsede.org/.

  2. http://www.penguincomputing.com/.

  3. http://blast.ncbi.nlm.nih.gov/Blast.cgi.

  4. https://slapos.cloud.univ-paris13.fr.

  5. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/bonjourgrid/software.cfg.

  6. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/bonjourgrid-client/software.cfg.

  7. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/boinc/software.cfg.

  8. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/boinc-client/software.cfg.

  9. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/condor/software.cfg.

  10. http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/grid-computing:/software/condor/instance-condor.cfg.

References

  1. Abbes H, Cérin C, Jemni M (2009) Bonjourgrid: orchestration of multi-instances of grid middlewares on institutional desktop grids. In: 23rd IEEE international symposium on parallel and distributed processing, IPDPS, pp 1–8

  2. Abbes H, Cérin C, Jemni M, Saad W (2010) Fault tolerance based on the publish-subscribe paradigm for the bonjourgrid middleware. In: Proceedings of the 11th IEEE/ACM international conference on grid computing GRID, pp 57–64

  3. Abidi L, Cérin C, Geldwerth-Feniger D, Lafaille M (2015) Cloud computing for e-sciences at université sorbonne paris cité. In: Cloudway workshop, service oriented and cloud computing—4th European conference, ESOCC 2015, Taormina, Italy, 15–17 September 2015, Proceedings of Springer CCIS series. vol 567

  4. Anderson DP (2004) BOINC: a system for public-resource computing and storage. In: 5th international workshop on grid computing GRID, pp 4–10

  5. Antoniu G, Costan A, Bigot J, Desprez F, Fedak G, Gault S, Pérez C, Simonet A, Tang B, Blanchet C, Terreux R, Bougé L, Briant F, Cappello F, Keahey K, Nicolae B, Suter F (2013) Scalable data management for map-reduce-based data-intensive applications: a view for cloud and hybrid infrastructures. IJCC 2(2/3):150–170

    Article  Google Scholar 

  6. Butt AR, Zhang R, Hu YC (2006) A self-organizing flock of condors. J Parallel Distrib Comput 66(1):145–161

    Article  MATH  Google Scholar 

  7. Chlumsky V, Klusácek D, Ruda M (2012) The extension of torque scheduler allowing the use of planning and optimization in grids. Comput Sci (AGH) 13(2):5–20

    Article  Google Scholar 

  8. Fedak G, He H, Cappello F (2009) BitDew: a data management and distribution service with multi-protocol file transfer and metadata abstraction. J Netw Comput Appl 32(5):961–975

    Article  Google Scholar 

  9. Foster I (2011) Globus online: accelerating and democratizing science through cloud-based services. IEEE Internet Comput 15(3):70–73

    Article  Google Scholar 

  10. Fox G, von Laszewski G, Diaz J, Keahey K, Fortes J, Figueiredo R, Smallen S, Smith W, Grimshaw A (2013) Futuregrid: a reconfigurable testbed for cloud, hpc, and grid computing. In: Contemporary high performance computing: from petascale toward exascale, CRC computational science. Chapman & Hall

  11. Grid’5000 (2015) Large-scale and Grid experimental testbed. http://www.grid5000.fr. Accessed 30 June 2015

  12. Grossman RL, Greenway M, Heath AP, Powell R, Suarez RD, Wells W, White KP, Atkinson MP, Klampanos IA, Alvarez HL, Harvey C, Mambretti J (2012) The design of a community science cloud: the open science data cloud perspective. In: SC companion: high performance computing. Networking storage and analysis, Salt Lake City, UT, USA, pp 1051–1057

  13. Juve G, Rynge M, Deelman E, Vöckler J, Berriman GB (2013) Comparing futuregrid, amazon ec2, and open science grid for scientific workflows. Comput Sci Eng 15(4):20–29

    Article  Google Scholar 

  14. Kang U, Chau DH, Faloutsos C (2012) Pegasus: Mining billion-scale graphs in the cloud. In: 2012 IEEE international conference on acoustics, speech and signal processing, ICASSP 2012, Kyoto, Japan, 25–30 March 2012, pp 5341–5344

  15. Kosar T, Livny M (2004) Stork: making data placement a first class citizen in the grid. In: 24th international conference on distributed computing systems ICDCS, pp 342–349

  16. Lifka D, Foster I, Mehringer S, Parashar M, Redfern P, Stewart C, Tuecke S (2013) XSEDE cloud survey report. Technical report, XSEDE Cloud Integration Investigation Team. http://www.cac.cornell.edu/technologies/XSEDECloudSurveyReport.pdf. Accessed 22 Oct 2014

  17. Nicolae B (2010) BlobSeer: towards efficient data storage management for large-scale, distributed systems. Theses, Université Rennes1

  18. Nicolae B, Antoniu G, Bougé L, Moise D, Carpen-Amarie A (2011) Blobseer: next-generation data management for large scale infrastructures. J Parallel Distrib Comput 71(2):169–184

    Article  Google Scholar 

  19. Redis Protocol (2015) Open source (BSD licensed) publish/subscribe and in-memory data structure store. http://redis.io/. Accessed 15 Feb 2015

  20. Saad W, Abbes H, Cérin C, Jemni M (2012) A self-configurable desktop grid system on-demand. In: Proceedings of the 2012 seventh international conference on P2P, parallel, grid, cloud and internet computing 3PGCIC, pp 196–203

  21. Saad W, Abbes H, Cérin C, Jemni M (2013) A data prefetching model for desktop grids and the condor use case. In: Proceedings of the 2013 12th IEEE international conference on trust, security and privacy in computing and communications TRUSTCOM ’13, pp 1065–1072

  22. Saad W, Abbes H, Cérin C, Jemni M (2014a) Designing and implementing a cloud-hosted saas for data movement and sharing with slapos. Int J Big Data Intell IJBDI 1(1/2):18–35

    Article  Google Scholar 

  23. Saad W, Abidi L, Abbes H, Cérin C, Jemni M (2014b) Wide area bonjourgrid as a data desktop grid: Modeling and implementation on top of redis. In: 26th IEEE international symposium on computer architecture and high performance computing, SBAC-PAD, pp 286–293

  24. Simonet A, Chard K, Fedak G, Foster IT (2015) Using active data to provide smart data surveillance to e-science users. In: 23rd Euromicro international conference on parallel, distributed, and Network-Based Processing, PDP 2015, Turku, Finland, 4–6 March 2015, pp 269–273

  25. Smets-Solanes JP, Cérin C, Courteaud R (2011) Slapos: A multi-purpose distributed cloud operating system based on an erp billing model. In: IEEE international conference on services computing SCC. IEEE, pp 765–766

  26. Steinberg D, Cheshire S (2005) Zero configuration networking: the definitive guide, 1st edn. O’Reilly Media, Inc., Sebastopol

    Google Scholar 

  27. Tang B, Moca M, Chevalier S, He H, Fedak G (2010) Towards mapreduce for desktop grid computing. In: 3PGCIC 2010, international conference on P2P, parallel, grid, cloud and internet computing, 4–6 Nov 2010. Fukuoka Institute of Technology, Fukuoka, pp 193–200

  28. White T (2015) Hadoop–the definitive guide: storage and analysis at internet scale, 4th edn. O’Reilly, Sebastopol (revised and updated)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Walid Saad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saad, W., Abbes, H., Cérin, C. et al. A self-organized volunteer Cloud for e-Science. J Supercomput 72, 1271–1290 (2016). https://doi.org/10.1007/s11227-015-1564-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-015-1564-z

Keywords

Navigation