skip to main content
10.1145/2608029.2608035acmconferencesArticle/Chapter ViewAbstractPublication PageshpdcConference Proceedingsconference-collections
research-article

HEP computing in a context-aware cloud environment

Published: 23 June 2014 Publication History

Abstract

This paper describes the use of a distributed cloud computing system for high energy physics (HEP) applications. The system is composed of IaaS clouds integrated into a unified infrastructure that has been in production for over two years. It continues to expand in scale and sites, encompassing more than twenty clouds on three continents. We are prototyping a new context-aware architecture that enables the virtual machines to make connections to both software and data repositories based on geolocation information. The new design will significantly enhance the ability of the system to scale to higher workloads and run data-intensive applications. We review the operation of the production system and describe our work towards a context-aware cloud system.

References

[1]
CANFAR: Canadian Advanced Network for Astronomical Research. https://wiki.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/canfar/
[2]
P. Armstrong, A. Agarwal, A. Bishop, A. Charbonneau, R. Desmarais, K. Fransham, N. Hill, I.Gable, S.Gaudet, S.Goliath, R.Impey, C. Leavett-Brown, J. Ouellete, M. Paterson, C. Pritchet, D. Penfold-Brown, W. Podaima, D. Schade, and R. Sobie. Cloud Scheduler: a resource manager for a distributed compute cloud. arXiv:1007.0050v1 {cs.DC}.
[3]
J. Blomer, P. Buncic, and T. Fuhrmann. CernVM-FS: delivering scientific software to globally distributed computing resources. In Proceedings of the first international workshop on Network-aware data management, NDM '11, pages 49--56, New York, NY, USA, 2011. ACM.
[4]
Frontier distributed database caching system. http://frontier.cern.ch/.
[5]
F. Furano et al. Dynamic federations: storage aggregation using open tools and protocols, Journal of Physics: Conference Series 396 (2012) 032042.
[6]
Katarzyna Keahey, Mauricio Tsugawa, Andrea Matsunaga, Jose Fortes, Sky Computing, IEEE Internet Computing, vol. 13, no. 5, pp. 43--51, 2009.
[7]
Nimbus: an open-source EC2/S3 compatible IaaS implementation. http://www.nimbusproject.org/.
[8]
H. Ohman et al. Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine. Proceedings of the CHEP 2013 Conference, Amsterdam 2013.
[9]
OpenStack: Open source software for building clouds. http://www.openstack.org.
[10]
Puppet Open Source. http://puppetlabs.com/puppet/puppet-open-source.
[11]
Root: a framework for the analysis of HEP data. http://root.cern.ch.
[12]
Randall Sobie, Ashok Agarwal, Ian Gable, Colin Leavett-Brown, Michael Paterson, Ryan Taylor, Andre Charbonneau, Roger Impey, and Wayne Podiama. 2013. HTC scientific computing in a distributed cloud environment. In Proceedings of the 4th ACM workshop on Scientific cloud computing (Science Cloud '13). ACM, New York, NY, USA, 45--52. DOI=10.1145/2465848.2465850 http://doi.acm.org/10.1145/2465848.2465850
[13]
D. Thain, T. Tannenbaum, and M. Livny. Distributed computing in practice: the HTCondor experience. Concurr. Comput. : Pract. Exper., 17(2--4):323--356, Feb. 2005.
[14]
Web Distributed Authoring and Versioning (WebDAV). http://tools.ietf.org/html/rfc4918.
[15]
I. Vukotic et al. . Data federation strategies for ATLASusing XRootD. Proceedings of the CHEP 2013 Conference, Amsterdam 2013.
[16]
I Gable et al. Dynamic web cache publishing for IaaS clouds using Shoal, Proceedings of the CHEP 2013 Conference, Amsterdam 2013. arXiv:1311.0058 {cs.DC}
[17]
Squids: a caching proxy for the Web. http://www.squid-cache.org.
[18]
M Vliet et al. Repoman: A Simple RESTful X.509 Virtual Machine Image Repository. Proceedings of the International Symposium on Grids and Clouds, Taipei, March 2011
[19]
XRootD: a high-performance protocol for accessing HEP data. http://xrootd.org

Index Terms

  1. HEP computing in a context-aware cloud environment

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ScienceCloud '14: Proceedings of the 5th ACM workshop on Scientific cloud computing
    June 2014
    70 pages
    ISBN:9781450329118
    DOI:10.1145/2608029
    • General Chairs:
    • Ioan Raicu,
    • Kate Keahey,
    • Program Chairs:
    • Kyle Chard,
    • Bogdan Nicolae
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 June 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tag

    1. distributed cloud computing

    Qualifiers

    • Research-article

    Conference

    HPDC'14
    Sponsor:

    Acceptance Rates

    ScienceCloud '14 Paper Acceptance Rate 8 of 17 submissions, 47%;
    Overall Acceptance Rate 44 of 151 submissions, 29%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 98
      Total Downloads
    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media