Skip to main content
SpringerLink
Log in

Belle-DIRAC Setup for Using Amazon Elastic Compute Cloud

Providing Homogeneous Access to Heterogeneous Computing Resources

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The Distributed Infrastructure with Remote Agent Control (DIRAC) software framework allows a user community to manage computing activities in a distributed environment. DIRAC has been developed within the Large Hadron Collider Beauty (LHCb) collaboration. After successful usage over several years, it is the final solution adopted by the experiment. The Belle experiment at the Japanese High Energy Accelerator Research Organization (KEK) has the purpose of studying matter/anti-matter asymmetries using B mesons. During its lifetime, Belle detector has collected about 5,000 terabytes of real and simulated data. The analysis of this data requires an enormous amount of computing intensive Monte Carlo simulation. The Belle II experiment, which recently published its technical design report, will produce 50 times more data. Therefore it is interested to determine if commercial computing clouds can reduce the total cost of the experiment’s computing solution. This paper describes the setup prepared to evaluate the performance and cost of this approach using real 2010 simulation tasks of the Belle experiment. The setup has been developed using DIRAC as the overall management tool to control both the tasks to be executed and the deployment of virtual machines using the Amazon Elastic Compute Cloud as service provider. At the same time, DIRAC is also used to monitor the execution, collect the necessary statistical data, and finally upload the results of the simulation to Belle resources on the Grid. The results of a first test using over 2000 days of cpu time show that over 90% efficiency in the use of the resources can easily be achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Tsaregorodtsev, A., Bargiotti, M., Brook, N., Casajus Ramo, A., Castellani, G., Charpentier, P., Cioffi, C., Closier, J., Graciani Diaz, R., Kuznetsov, G., Li, Y.Y., Nandakumar, R., Paterson, S., Santinelli, R., Smith, A.C., Miguelez, M.S., Jimenez, S.G.: DIRAC: a community Grid solution. J. Phys. Conf. Ser. 119, 062048 (2008). http://stacks.iop.org/1742-6596/119/i=6/a=062048

    Article  Google Scholar 

  2. Amato, S., et al.: LHCb Technical Proposal. Tech. Rep. LHCb CERN-LHCC-98-04. LHCC-P-4 (1998)

  3. Antunes-Nobrega, R., et al.: (LHCb) LHCb Technical Design Report: Reoptimized Detector Design And Performance. Tech. Rep. LHCb CERN-LHCC-2003-030 (2003)

  4. Tsaregorodtsev, A., et al.: DIRAC: distributed infrastructure with remote agent control. In: Proc. Conference for Computing in High Energy and Nuclear Physics. La Jolla, California (Preprint, cs/0306060v1, 2003)

  5. Antunes-Nobrega, R., et al.: (LHCb) LHCb TDR Computing Technical Design Report. Tech. Rep. LHCb CERN-LHCC-2005-019 (2005)

  6. CERN Advanced Storage Manager. http://castor.web.cern.ch/castor/ (2010). Accessed 10 November 2010

  7. Smith, A.C., Tsaregorodtsev, A.: DIRAC: reliable data management for LHCb. J. Phys. Conf. Ser. 119, 062045 (2008). http://stacks.iop.org/1742-6596/119/i=6/a=062045

    Article  Google Scholar 

  8. Abashian, A., et al.: (Belle). Nucl. Instrum. Methods A479, 117–232 (2002)

    Google Scholar 

  9. Luminosity at B Factories. http://belle.kek.jp/bdocs/lumi_belle.png

  10. Abe, T., Adachi, I., Adamczyk, K., Ahn, S., Aihara, H., Akai, K., Aloi, M., Andricek, L., Aoki, K., Arai, Y., et al.: arxiv:1011.0352 (Preprint, 2010)

  11. Sevior, M., Fifield, T., Katayama, N.: Belle Monte-Carlo production on the Amazon EC2 cloud. J. Phys. Conf. Ser. 219, 012003 (2010). http://stacks.iop.org/1742-6596/219/i=1/a=012003

    Article  Google Scholar 

  12. Evans, L., Bryant, P., eds.: LHC machine. JINST 3, S08001 (2008)

  13. Aamodt, K., et al.: (ALICE). JINST 0803, S08002 (2008)

    Google Scholar 

  14. Aad, G., et al.: (ATLAS). JINST 3, S08003 (2008)

    Google Scholar 

  15. Adolphi, R., et al.: (CMS). JINST 0803, S08004 (2008)

    Google Scholar 

  16. Alves, A.A., et al.: (LHCb). JINST 3, S08005 (2008)

    Google Scholar 

  17. Chase, J.S., Irwin, D.E., Grit, L.E., Moore, J.D., Sprenkle, S.E.: Dynamic virtual clusters in a Grid site manager. International symposium on high-performance distributed computing, pp. 90–100 (2003). ISSN 1082-8907

  18. Foster, I., Freeman, T., Keahy, K., Scheftner, D., Sotomayer, B., Zhang, X.: Virtual clusters for Grid communities. Sixth IEEE International Symposium on Cluster Computing and the Grid, vol. 1, pp. 513–520 (2006)

  19. Murphy, M.A., Kagey, B., Fenn, M., Goasguen, S.: Dynamic provisioning of virtual organization clusters. In: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID ’09. IEEE Computer Society, Washington, pp 364–371 (2009). ISBN 978-0-7695-3622-4. doi:10.1109/CCGRID.2009.37

  20. Nishimura, H., Maruyama, N., Matsuoka, S.: Virtual clusters on the fly - fast, scalable, and flexible installation. IEEE international symposium on cluster computing and the Grid, pp. 549–556 (2007)

  21. Emeneker, W., Jackson, D., Butikofer, J., Stanzione, D.: Dynamic virtual clustering with xen and moab. In: Min, G., Di Martino, B., Yang, L., Guo M., Ruenger, G. (eds.) Frontiers of High Performance Computing and Networking ISPA 2006 Workshops. Lecture Notes in Computer Science, vol. 4331, pp. 440–451. Springer, Berlin/Heidelberg (2006). doi:10.1007/11942634_46.

  22. Emeneker, W., Stanzione, D.: Dynamic virtual clustering. In: Proceedings of the 2007 IEEE International Conference on Cluster Computing, CLUSTER ’07, pp 84–90. IEEE Computer Society, Washington (2007). ISBN 978-1-4244-1387-4. doi:10.1109/CLUSTR.2007.4629220

  23. Worker Nodes on Demand. http://web.infn.it/wnodes/index.php (2010). Accessed 10 Nov 2010

  24. StratusLab Deliverable 2.1. http://stratuslab.eu/lib/exe/fetch.php?media=documents:stratuslab-d2.1-1.2.pdf (2010). Accessed 10 Nov 2010

  25. Bégin, M.E.: An EGEE Comparative Study: Grids and Clouds—Evolution or Revolution Tech. Rep. CERN—Engeneering and Equipment Data Management Service (2008)

  26. StratusLab. http://www.stratuslab.eu (2010). Accessed 10 November 2010

  27. Venus-C. http://www.venus-c.eu (2010). Accessed 10 Nov 2010

  28. RESEVOIR. http://62.149.240.97 (2010). Accessed 10 Nov 2010

  29. Clobi. http://code.google.com/p/clobi/ (2010). 10 Nov 2010

  30. Maier, A.: Ganga, a job management and optimising tool. J. Phys. Conf. Ser. 119, 072021 (2008). http://stacks.iop.org/1742-6596/119/i=7/a=072021

    Article  Google Scholar 

  31. Casajus, A., Graciani, R., the Lhcb Dirac Team: DIRAC distributed secure framework. J. Phys. Conf. Ser. 219, 042033 (2010). http://stacks.iop.org/1742-6596/219/i=4/a=042033

    Article  Google Scholar 

  32. Housley, R., Polk, W., Ford, W., Solo, D.: [RFC3280] Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile. RFC United States (2002)

  33. Security/ProxyCertTypes—Globus. http://dev.globus.org/wiki/Security/ProxyCertTypes#Legacy_Proxy_Certificates (2010). Accessed 10 Nov 2010.

  34. OpenSSL: The Open Source toolkit for SSL/TLS. http://www.openssl.org/ (2010). Accessed 10 Nov 2010

  35. Casajus, A., Graciani, R., Paterson, S., Tsaregorodtsev, A., the Lhcb Dirac Team: DIRAC pilot framework and the DIRAC Workload Management System. J. Phys. Conf. Ser. 219, 062049 (2010). http://stacks.iop.org/1742-6596/219/i=6/a=062049

    Article  Google Scholar 

  36. Lightweight Middleware for Grid Computing. http://glite.web.cern.ch/glite/ (2010). Accessed 10 Nov 2010

  37. Casajus Ramo, A., Sapunov, M.: DIRAC: Secure web user interface. J. Phys. Conf. Ser. 219, 082004 (2010). http://stacks.iop.org/1742-6596/219/i=8/a=082004

    Article  Google Scholar 

  38. PostgreSQL 8.1.21 Documentation. http://www.postgresql.org/docs/8.1/ (2010). Accessed 10 Nov 2010

  39. Scientific Linux. https://www.scientificlinux.org/ (2010). Accessed 10 Nov 2010

Download references

Author information

Authors and Affiliations

  1. University of Barcelona, Barcelona, Spain

    Ricardo Graciani Diaz, Adria Casajus Ramo & Ana Carmona Agüero

  2. University of Melbourne, Melbourne, Victoria, Australia

    Thomas Fifield & Martin Sevior

Authors
  1. Ricardo Graciani Diaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adria Casajus Ramo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ana Carmona Agüero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Fifield
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martin Sevior
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Graciani Diaz.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Graciani Diaz, R., Casajus Ramo, A., Carmona Agüero, A. et al. Belle-DIRAC Setup for Using Amazon Elastic Compute Cloud. J Grid Computing 9, 65–79 (2011). https://doi.org/10.1007/s10723-010-9175-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-010-9175-7

Keywords

Search

Navigation