Skip to main content

Advertisement

SpringerLink
Log in

Cloud computing and virtualization within the regional climate model and evaluation system

  • Research Article
  • Published:
Earth Science Informatics Aims and scope Submit manuscript

Abstract

The Regional Climate Model Evaluation System (RCMES) facilitates the rapid, flexible inclusion of NASA observations into climate model evaluations. RCMES provides two fundamental components. A database (RCMED) is a scalable point-oriented cloud database used to elastically store remote sensing observations and to make them available using a space time query interface. The analysis toolkit (RCMET) is a Python-based toolkit that can be delivered as a cloud virtual machine, or as an installer package deployed using Python Buildout to users in order to allow for temporal and spatial regridding, metrics calculation (RMSE, bias, PDFs, etc.) and end-user visualization. RCMET is available to users in an “offline”, lone scientist mode based on a virtual machine dynamically constructed with model outputs and observations to evaluate; or on an institution’s computational cluster seated close to the observations and model outputs. We have leveraged RCMES within the content of the Coordinated Regional Downscaling Experiment (CORDEX) project, working with the University of Cape Town and other institutions to compare the model output to NASA remote sensing data; in addition we are also working with the North American Regional Climate Change Assessment Program (NARCCAP). In this paper we explain the contribution of cloud computing to RCMES’s specifically describing studies of various cloud databases we evaluated for RCMED, and virtualization toolkits for RCMET, and their potential strengths in delivering user-created dynamic regional climate model evaluation virtual machines for our users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. We were curious if running the benchmark locally without any nodes would result in a quicker response time since the delay appeared to be in the mapping stage.

References

  • Banker K (2011) MongoDB in Action. Manning Publications Co., Greenwich, CT, USA

  • Capriolo E, Wampler D, Rutherglen J (2012) Programming hive. O’Reilly

  • Cinquini L, Crichton D, Mattmann C, Harney J, Shipman G, Wang F, Ananthakrishnan R, Miller N, Denvil S, Morgan M, Pobre Z, Bell G, Drach B, Williams D, Kershaw P, Pascoe S, Gonzalez E, Fore S, Schweitzer R (2012) The Earth System Grid Federation: An Open Infrastructure for Access to Distributed Geospatial Data. In Proceedings of the 8th IEEE International Conference on eScience 2012, Chicago, IL, October 8–12

  • Cornillon P, Gallagher J, Sgouros T (2003) OPeNDAP: Accessing data in a distributed, heterogeneous environment. Data Sci J 2:164–174

    Article  Google Scholar 

  • Crichton D, Mattmann C, Cinquini L, Braverman A, Waliser D, Hart A, Goodale C, Lean P (2012) Sharing Satellite Observations with the Climate Modeling Community: Software and Architecture. IEEE Software, Vol. 29, No. 5., September/October, pp 63–71

  • Fielding RT, Taylor RN (2002) Principled design of the modern Web architecture. ACM Trans Internet Technol (TOIT) 2(2):115–150

    Article  Google Scholar 

  • B Fortner (1998) “HDF: The Hierarchical Data Format,” Dr Dobb’s J. Software Tools and Pro- fessional Programming, www.drdobbs.com/database/184410553

  • Giorgi F, Jones C, Asrar G (2009) Addressing climate information needs at the regional level: the CORDEX framework. WMO Bulletin 58(3):175–183

    Google Scholar 

  • Hart A, Goodale C, Mattmann C, Zimdars P, Crichton D, Lean P, Kim J, Waliser D (2011) A Cloud-Enabled Regional Climate Model Evaluation System. In Proceedings of the ICSE 2011 Workshop on Software Engineering for Cloud Computing - SECLOUD, Honolulu, HI, May 22

  • Hüttermann M (2012) Infrastructure as Code. In DevOps for Developers, Part IV. Springer, pp 135–156

  • Mattmann C, Crichton D, Medvidovic N, Hughes S (2006) A Software Architecture-Based Framework for Highly Distributed and Data Intensive Scientific Applications. In Proceedings of the 28th International Conference on Software Engineering (ICSE06), Software Engineering Achievements Track, pp 721–730, Shanghai, China, May 20th-28th

  • Mattmann C, Freeborn D, Crichton D, Foster B, Hart A, Woollard D, Hardman S, Ramirez P, Kelly S, Chang AY, Miller CE (2009) A reusable process control system framework for the orbiting carbon observatory and NPP sounder PEATE missions. In proceedings of the 3rd IEEE Intl Conference on Space Mission Challenges for Information Technology (SMC-IT 2009), pp 165–172, July 19–23

  • Mearns L, Arritt R, Biner S, Bukovsky MS, McGinnis S, Sain S, Caya D, Correia J, Flory D, Gutowski W, Takle ES, Jones R, Leung R, M-Okia W, McDaniel L, Nunes AMB, Qian Y, Roads J, Sloan L, Synder M. The North American regional climate change assessment program: overview of phase I results. Bull Amer Meteor Soc 93: 1337–1362

  • Obe R, Hsu L (2011) PostGIS in Action. Manning Publications Co., Greenwich, CT, USA

  • Rew RK, Davis GP (1990) NetCDF: An Interface for Scientific Data Access. IEEE Comput Graph Appl 10(4):76–82

    Article  Google Scholar 

  • Rew RK et al (2007) “The CF Conventions: Gover- nance and Community Issues in Establishing Standards for Representing Climate, Forecast, and Observational Data,” slide presentation, Am. Geophysical Union Fall Meeting, abstract #IN52A-07

  • Virtual Appliances (2012) VMWare, http://www.vmware.com/technical-resources/virtualization-topics/virtual-appliances/faqs

  • T. White (2011) Hadoop: the Definitive Guide. 2nd Edition, O’Reilly

  • Whitehall K, Mattmann C, Waliser D, Kim J, Goodale C, Hart A, Ramirez P, Zimdars P, Crichton D, Jenkins G, Jones C, Asrar G, Hewitson B (2012) Building model evaluation and decision support capacity for CORDEX. WMO Bulletin, Vol. 61, No. 2, pp 29–34, http://www.wmo.int/pages/publications/bulletin_en/61_2_cordex_en.html

Download references

Acknowledgments

This work was conducted at the Jet Propulsion Laboratory, managed by the California Institute of Technology, for the National Aeronautics and Space Administration. The research reported on herein was sponsored by the NASA Advanced Information Systems (AIST) program (AIST-QRS-12-0002-T). Thanks are due to the RCMES team including Duane Waliser, Jinwon Kim, Cameron Goodale, Andrew Hart, Paul Ramirez, Paul Zimdars, Kim Whitehall, Jesslyn Whittell and Dan Crichton. The author also wishes to thank Michael Seablom for his support in this effort.

Author information

Authors and Affiliations

  1. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA

    Chris A. Mattmann, Duane Waliser, Cameron Goodale, Andrew Hart, Paul Ramirez, Dan Crichton, Paul Zimdars, Maziyar Boustani, Kyo Lee, Paul Loikith & Kim Whitehall

  2. UCLA JIFRESSE, Los Angeles, CA, USA

    Chris A. Mattmann, Duane Waliser & Jinwon Kim

  3. Howard University, Washington, DC, USA

    Kim Whitehall

  4. University of Southern California, Los Angeles, CA, USA

    Chris A. Mattmann

  5. University of Cape Town, South Africa, Cape Town, South Africa

    Chris Jack & Bruce Hewitson

Authors
  1. Chris A. Mattmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Duane Waliser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinwon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cameron Goodale
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrew Hart
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Paul Ramirez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dan Crichton
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Paul Zimdars
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Maziyar Boustani
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Kyo Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Paul Loikith
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kim Whitehall
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Chris Jack
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Bruce Hewitson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chris A. Mattmann.

Additional information

Communicated by: H. A. Babaie

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mattmann, C.A., Waliser, D., Kim, J. et al. Cloud computing and virtualization within the regional climate model and evaluation system. Earth Sci Inform 7, 1–12 (2014). https://doi.org/10.1007/s12145-013-0126-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12145-013-0126-2

Keywords

Search

Navigation