Skip to main content

Advertisement

Springer Nature Link
Log in

CyberConnector: a service-oriented system for automatically tailoring multisource Earth observation data to feed Earth science models

  • Research Article
  • Published:
Earth Science Informatics Aims and scope Submit manuscript

Abstract

Feeding multisource Earth observation (EO) data into Earth science models (ESM) remains a daunting challenge. This paper presents a service-oriented approach as an alternative solution. It uses geospatial web services to process the EO data and geoprocessing workflow for automation. Different from existing approaches, it takes advantage of virtual data products (VDP) to release modelers from intensive data processing. It can directly connect ESMs to public EO sources via Cyberinfrastructure. A prototype called CyberConnector is implemented. CyberConnector supports intuitive building of VDP, automatic execution of workflows and effortless retrieval of model-ready input files. We used it to stream multiple datasets to several ESMs including finite-volume coastal ocean model (FVCOM) and cloud-resolving model (CRM). The results show that CyberConnector can truly benefit modelers on time saving and effort minimizing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Allison ML, Ahern T, Arctur D, Bowring J, Crane G, DeLuca C, Fox G, Hood C, Leetaru H, Lehnert K (2012) EarthCube governance roadmap: documentation, Research, and Recommendations, Version 20 August

  • Amanchi SK, Durbha SS, King RL, Bheemireddy S, Younan NH (2009) Mobile computing and sensor web services for coastal buoys. Geoscience and Remote Sensing Symposium,2009 I.E. International, IGARSS 2009 5:465–468

  • Apache (2006) Apache Axis 1.4. https://axis.apache.org/axis/

  • Atkins D (2003) Revolutionizing science and engineering through cyberinfrastructure: Report of the National Science Foundation blue-ribbon advisory panel on cyberinfrastructure http://hdl.handle.net/10150/106224

  • Benlian A, Hess T (2011) Opportunities and risks of software-as-a-service: findings from a survey of IT executives. Decis Support Syst 52(1):232–246

    Article  Google Scholar 

  • Berners-Lee TJ (1992) The world-wide web. Computer Networks and ISDN Systems 25(4):454–459

    Article  Google Scholar 

  • Berners-Lee T, Hendler J, Lassila O, (2001) The Semantic Web. Sci Am 284 (5):34–43

  • Bröring A, Jürrens EH, Jirka S, Stasch C (2009) Development of sensor web applications with open source software. In: First Open Source GIS UK Conference (OSGIS 2009). Nottingham, UK

  • Buxmann P, Hess T, Lehmann D (2008) Software as a service. Wirtschaftsinformatik 50(6):500–503

    Article  Google Scholar 

  • Chassignet EP, Hurlburt HE, Smedstad OM, Halliwell GR, Hogan PJ, Wallcraft AJ, Baraille R, Bleck R (2007) The HYCOM (hybrid coordinate ocean model) data assimilative system. J Mar Syst 65(1):60–83

    Article  Google Scholar 

  • Chen C, Beardsley RC, Cowles G (2006) An unstructured grid, finite-volume coastal ocean model (FVCOM) system. Oceanography 19(1):78–89

    Article  Google Scholar 

  • Chen C, Beardsley RC, Cowles GW, Qi J, Lai Z, Gao G, Stuebe DA, Xu Q, Xue P, Ge J (2012) An unstructured-grid, finite-volume community ocean model: FVCOM user manual. Sea Grant College Program, Massachusetts Institute of Technology. http://fvcom.smast.umassd.edu/wp-content/uploads/2013/11/MITSG_12-25.pdf

  • Chen A, Di L, Wei Y, Bai Y, Liu Y (2009a) Use of grid computing for modeling virtual geospatial products. Int J Geogr Inf Sci 23(5):581–604

    Article  Google Scholar 

  • Chen N, Di L, Yu G, Gong J (2010) Geo-processing workflow driven wildfire hot pixel detection under sensor web environment. Comput Geosci 36(3):362–372

    Article  Google Scholar 

  • Chen N, Di L, Yu G, Min M (2009b) A flexible geospatial sensor observation service for diverse sensor data based on web service. ISPRS J Photogramm Remote Sens 64(2):234–242

    Article  Google Scholar 

  • Chen C, Huang H, Beardsley RC, Xu Q, Limeburner R, Cowles GW, Sun Y, Qi J, Lin H (2011) Tidal dynamics in the Gulf of Maine and New England shelf: an application of FVCOM. J Geophys Res Oceans 116(C12010):1–14

  • Chen C, Liu H, Beardsley RC (2003) An unstructured grid, finite-volume, three-dimensional, primitive equations ocean model: application to coastal ocean and estuaries. J Atmos Ocean Technol 20(1):159–186

    Article  Google Scholar 

  • Chen C, Qi J, Li C, Beardsley RC, Lin H, Walker R, Gates K (2008) Complexity of the flooding/drying process in an estuarine tidal-creek salt-marsh system: an application of FVCOM. J Geophys Res Oceans 113(C7052):1–21

  • Ciesielski PE, Yu H, Johnson RH, Yoneyama K, Katsumata M, Long CN, Wang J, Loehrer SM, Young K, Williams SF (2014) Quality-controlled upper-air sounding dataset for DYNAMO/CINDY/AMIE: Development and corrections. J Atmos Ocean Technol 31(4):741–764

    Article  Google Scholar 

  • Cyberinfrastructure Council (2007) Cyberinfrastructure vision for 21st century discovery. National Science Foundation, https://www.nsf.gov/pubs/2007/nsf0728/nsf0728.pdf

  • Coward D, Yoshida Y (2003) Java(tm) servlet specification version 2.4. https://jcp.org/aboutJava/communityprocess/final/jsr154/index.html

  • Delin KA, Jackson SP (2001) Sensor web: a new instrument concept. In: Symposium on integrated optics, p 1–9

  • Delin KA, Jackson SP, Some RR (1999) Sensor webs. NASA Tech Briefs 23:80

    Google Scholar 

  • Di L (2007) Geospatial sensor web and self-adaptive Earth predictive systems (SEPS). In: Proceedings of the Earth science Technology office (ESTO)/advanced information system Technology (AIST) sensor web principal investigator (PI) Meeting, San Diego, USA. p 1–4

  • Di L (2013) Engaging the Atmospheric Cloud/Aerosol/Composition Community Executive Summary: Workshop Results http://earthcube.org/sites/default/files/doc-repository/AtmoCloudAerosolComp_EndUserWorkshop_ExecSummary.pdf

  • Díaz L, Granell C, Gould M, Huerta J (2011) Managing user-generated information in geospatial cyberinfrastructures. Futur Gener Comput Syst 27(3):304–314

  • Egenhofer MJ (2002) Toward the semantic geospatial web. In: Proceedings of the 10th ACM international symposium on Advances in geographic information systems. Mclean, Virginia 1–4

  • Fonseca F, Sheth A (2002) The geospatial semantic web. In: Wilson JP, Fotheringham SA (eds) The handbook of geographic information science. John Wiley & Sons, Oxford, pp 367–376

    Google Scholar 

  • Fraternali P (1999) Tools and approaches for developing data-intensive web applications: a survey. ACM Computing Surveys (CSUR) 31(3):227–263

    Article  Google Scholar 

  • Fridlind A, Ackerman A, Chaboureau JP, Fan J, Grabowski WW, Hill A, Jones T, Khaiyer M, Liu G, Minnis P (2012) A comparison of TWP-ICE observational data with cloud-resolving model results. J Geophys Res Atmos 117(D5204):1–28

  • Gaber N, Laniak G, Linker L (2008) Integrated modeling for integrated environmental decision making. US Environmental Protection Agency, Washington, DC

    Google Scholar 

  • Geller GN, Melton F (2008) Looking forward: applying an ecological model web to assess impacts of climate change. Biodiversity 9(3–4):79–83

    Article  Google Scholar 

  • GEO (2013) GEOSS future products workshop summary and recommendations report. GEOSS Future Products Workshop. NOAA Science Center, Silver Spring, Maryland https://portal.opengeospatial.org/files/?artifact_id=53993

    Google Scholar 

  • Goodall JL, Horsburgh JS, Whiteaker TL, Maidment DR, Zaslavsky I (2008) A first approach to web services for the National Water Information System. Environ Model Softw 23(4):404–411

  • Goodall JL, Robinson BF, Castronova AM, (2011) Modeling water resource systems using a serviceoriented computing paradigm. Environ Model Softw 26(5):573–582

  • Grabowski WW, Wu X, Moncrieff MW (1996) Cloud-resolving modeling of tropical cloud systems during phase III of GATE. Part I: two-dimensional experiments. J Atmos Sci 53(24):3684–3709

    Article  Google Scholar 

  • Granell C, Díaz L, Gould, M (2010) Service-oriented applications for environmental models: Reusable geospatial services. Environ Model Softw 25(2):182–198

  • Hazzard E (2011) Openlayers 2.10 beginner’s guide. Packt publishing ltd. http://openlayers.org/

  • Hendler J (2001) Agents and the semantic web. IEEE Intell Syst 2:30–37

    Article  Google Scholar 

  • Hey T, Trefethen AE (2005) Cyberinfrastructure for e-science. Science 308(5723):817–821

    Article  Google Scholar 

  • Horsburgh JS, Tarboton DG, Piasecki M, Maidment DR, Zaslavsky I, Valentine D, Whitenack T (2009) An integrated system for publishing environmental observations data. Environ Model Softw 24(8):879–888

  • Jirka S, Bröring A, Stasch C (2009) Applying OGC sensor web enablement to risk monitoring and disaster management. In: GSDI 11 world Conference, Rotterdam, Netherlands

  • Johnson RH, Ciesielski PE (2013) Structure and properties of Madden–Julian oscillations deduced from DYNAMO sounding arrays. J Atmos Sci 70(10):3157–3179

    Article  Google Scholar 

  • Kerr PC, Donahue AS, Westerink JJ, Luettich RA, Zheng L, Weisberg RH, Huang Y, Wang H, Teng Y, Forrest D (2013) US IOOS coastal and ocean modeling testbed: inter-model evaluation of tides, waves, and hurricane surge in the Gulf of Mexico. J Geophys Res Oceans 118(10):5129–5172

    Article  Google Scholar 

  • Khairoutdinov MF, Randall DA (2001) A cloud resolving model as a cloud parameterization in the NCAR Community climate system model: preliminary results. Geophys Res Lett 28(18):3617–3620

    Article  Google Scholar 

  • Khairoutdinov M, Randall D, DeMott C (2005) Simulations of the Atmospheric general circulation using a cloud-resolving model as a Superparameterization of physical processes. J Atmos Sci 62(7):2136–2154

    Article  Google Scholar 

  • Kofler M (2001) What is MySQL?, MySQL Apress, 3–19

  • Lassila O, Swick RR (1998) Resource description framework (RDF) model and syntax specification. W3C Working Draft. https://www.w3.org/TR/1998/WD-rdf-syntax-19980216/

  • Martinek, P., Szikora, B., 2007. Semantic execution of BPEL processes. Advances in Information Systems Development, Vol 1, 361–367, 550.

  • Mesinger F, DiMego G, Kalnay E, Mitchell K, Shafran PC, Ebisuzaki W, Jovic D, Woollen J, Rogers E, Berbery EH (2006) North American regional reanalysis. Bull Am Meteorol Soc 87(3):343–360

    Article  Google Scholar 

  • NASA (2014) MODIS data products table https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table

  • Nativi S, Mazzetti P, Geller GN (2013) Environmental model access and interoperability: the GEO model web initiative. Environ Model Softw 39:214–228

    Article  Google Scholar 

  • Norris JM (2010) US Geological Survey Streamgage Operation and Maintenance Cost Evaluation... from the National Streamflow Information Program. US Geological Survey

  • OSGEO (2008) GDAL-OGR: geospatial data abstraction library/simple features library software http://www.gdal.org

  • Rew R, Davis G, Emmerson S, Davies H (1997) NetCDF user’s guide for C. Unidata Program Center

    Google Scholar 

  • Richard SM, Pearthree G, Aufdenkampe AK, Cutcher-Gershenfeld J, Daniels M, Gomez B, Kinkade D, Percivall G (2014) Community-developed Geoscience cyberinfrastructure. Eos, Trans American Geophysical Union 95(20):165–166

    Article  Google Scholar 

  • Roman D, Schade S (2009) Model as a service (MaaS). In: AGILE workshop: grid technologies for geospatial applications. Hannover, Germany

  • Santos E, Poco J, Wei Y, Liu S, Cook B, Williams DN, Silva CT (2013) UV-CDAT: analyzing climate datasets from a user’s perspective. Computing in Science & Engineering 15(1):94–103

    Article  Google Scholar 

  • Schmunck RB (2002) Panoply netCDF, HDF and GRIB Data Viewer http://www.giss.nasa.gov/tools/panoply/

  • Shen Y, Carriere L, Nadeau D, Potter G, Peters J, Winter E, Cinquini L, Blodgett D, McInerney M, (2014) Combining ESGF data node with its complementary data services at the NASA Center for climate simulation. In: AGU Fall Meeting Abstracts. 3771

  • Sun Z, Peng C, Deng M, Chen A, Yue P, Fang H, Di L (2014) Automation of customized and near-real-time vegetation condition index generation through cyberinfrastructure-based geoprocessing workflows. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7(11):4512–4522

    Article  Google Scholar 

  • Sun Z, Yue P, Hu L, Gong J, Zhang L, Lu X (2013) GeoPWProv: interleaving map and faceted metadata for provenance visualization and navigation. IEEE Trans Geosci Remote Sens 51(11):5131–5136

    Article  Google Scholar 

  • Sun Z, Yue P, Lu X, Zhai X, Hu L (2012) A task ontology driven approach for live geoprocessing in a service oriented environment. Trans GIS 16(6):867–884

    Article  Google Scholar 

  • Tao W, Moncrieffg M (2008) Cloud-system resolving models: status and prospects http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20080023285.pdf

  • Team RC (2000) R language definition ftp://155.232.191.133/cran/doc/manuals/r-devel/R-lang.pdf

  • Tsou MH, Kim I-H (2011) Creating GIS simulation models on a TeraGrid-enabled geospatial web portal: A demonstration of geospatial cyberinfrastructure. Advances in web-based GIS, mapping services and applications, 55–70

  • Turner M, Budgen D, Brereton P (2003) Turning software into a service. Computer 36(10):38–44

    Article  Google Scholar 

  • Wang S (2010) A CyberGIS framework for the synthesis of cyberinfrastructure, GIS, and spatial analysis. Ann Assoc Am Geogr 100(3):535–557

    Article  Google Scholar 

  • Wang S, Anselin L, Bhaduri B, Crosby C, Goodchild MF, Liu Y, Nyerges TL (2013) CyberGIS software: a synthetic review and integration roadmap. Int J Geogr Inf Sci 27(11):2122–2145

    Article  Google Scholar 

  • Williams D (2015) The Earth system grid federation (ESGF): climate science infrastructure for large-scale data management and dissemination. In: AGU Fall Meeting Abstracts http://adsabs.harvard.edu/abs/2015AGUFMIN11G..02W

  • Wright DJ, Wang S (2011) The emergence of spatial cyberinfrastructure. Proc Natl Acad Sci 108(14):5488–5491

    Article  Google Scholar 

  • Wu X, Deng L (2013) Comparison of moist static energy and budget between the GCM-simulated Madden–Julian oscillation and observations over the Indian Ocean and western Pacific. J Clim 26(14):4981–4993

    Article  Google Scholar 

  • Wu X, Li X (2008) A review of cloud-resolving model studies of convective processes. Adv Atmos Sci 25(2):202–212

    Article  Google Scholar 

  • Wu X, Liang X, Park S (2007) Cloud-resolving model simulations over the ARM SGP. Mon Weather Rev 135(8):2841–2853

    Article  Google Scholar 

  • Wu X, Moncrieff MW, Emanuel KA (2000) Evaluation of large-scale forcing during TOGA COARE for cloud-resolving models and single-column models. J Atmos Sci 57(17):2977–2985

    Article  Google Scholar 

  • Xue P, Chen C, Qi J, Beardsley RC, Tian R, Zhao L, Lin H (2014) Mechanism studies of seasonal variability of dissolved oxygen in Mass Bay: a multi-scale FVCOM/UG-RCA application. J Mar Syst 131:102–119

    Article  Google Scholar 

  • Yang C, Nebert D, Taylor DF (2011) Establishing a sustainable and cross-boundary geospatial cyberinfrastructure to enable polar research. Comput Geosci 37(11):1721–1726

    Article  Google Scholar 

  • Yang C, Raskin R, Goodchild M, Gahegan M (2010) Geospatial cyberinfrastructure: past, present and future. Comput Environ Urban Syst 34(4):264–277

    Article  Google Scholar 

  • Yoneyama K, Zhang C, Long CN (2013) Tracking pulses of the Madden-Julian oscillation. Bull Am Meteorol Soc 94(12):1871–1891

    Article  Google Scholar 

  • Yu G, Zhao P, Di L, Chen A, Deng M, Bai Y (2012) BPELPower-A BPEL execution engine for geospatial web services. Comput Geosci 47(0):87–101

    Article  Google Scholar 

  • Yue P, Di L, Yang W, Yu G, Zhao P (2007) Semantics-based automatic composition of geospatial web service chains. Comput Geosci 33(5):649–665

    Article  Google Scholar 

  • Yue P, Di L, Zhao P, Yang W, Yu G, Wei Y (2006) Semantic augmentations for geospatial catalogue service. In: 2006 I.E. International Geoscience and Remote Sensing Symposium (IGARSS06), p 3486–3489

  • Yue P, Gong J, Di L, He L, Wei Y (2011) Integrating semantic web technologies and geospatial catalog services for geospatial information discovery and processing in cyberinfrastructure. GeoInformatica 15(2):273–303

    Article  Google Scholar 

  • Yue P, He L (2009).Geospatial data provenance in cyberinfrastructure. In: 2009 17th international Conference on Geoinformatics, p 1–4.

  • Zhang T, Tsou M-H (2009) Developing a grid-enabled spatial web portal for internet GIServices and geospatial cyberinfrastructure. Int J Geogr Inf Sci 23(5):605–630

    Article  Google Scholar 

  • Zhang C, Zhao T, Li W, Osleeb JP (2010) Towards logic-based geospatial feature discovery and integration using web feature service and geospatial semantic web. Int J Geogr Inf Sci 24(6):903–923

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported partially by grants from NSF EarthCube (Grant # ICER-1440294, PI: Dr. Liping Di), the U.S. Department of Energy (Grant # DE-NA0001123, PI: Dr. Liping Di) and OGC Testbed 12.

Author information

Authors and Affiliations

  1. Center for Spatial Information Science and Systems, George Mason University, 4400 University Drive, MSN 6E1, Fairfax, VA, 22030, USA

    Ziheng Sun, Liping Di, Daniel Q. Tong, Chen Zhang, Hui Fang, Eugene Yu & Li Lin

  2. Department of Oceanography and Coastal Science, Louisiana State University, Baton Rouge, LA, USA

    Haosheng Hao

  3. Department of Geological and Atmospheric Sciences, Iowa State University, Ames, IA, USA

    Xiaoqing Wu & Cora Virgei

  4. International School of Software, Wuhan University, Wuhan, Hubei, People’s Republic of China

    Xicheng Tan

  5. LIESMARS, Wuhan University, Wuhan, Hubei, People’s Republic of China

    Peng Yue

Authors
  1. Ziheng Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Liping Di
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Haosheng Hao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaoqing Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Daniel Q. Tong
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Chen Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Cora Virgei
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Hui Fang
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Eugene Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Xicheng Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Peng Yue
    View author publications

    You can also search for this author inPubMed Google Scholar

  12. Li Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Liping Di.

Additional information

Communicated by: H. A. Babaie

Appendix

Appendix

Table 1

Table 1 A list of FVCOM input files
Full size table

Table 2

Table 2 A list of CRM input files
Full size table

Table 3

Table 3 The pre-installed services in CyberConnector (84 in total)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Z., Di, L., Hao, H. et al. CyberConnector: a service-oriented system for automatically tailoring multisource Earth observation data to feed Earth science models. Earth Sci Inform 11, 1–17 (2018). https://doi.org/10.1007/s12145-017-0308-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12145-017-0308-4

Keywords