skip to main content
10.1145/3491418.3535148acmconferencesArticle/Chapter ViewAbstractPublication PagespearcConference Proceedingsconference-collections
research-article
Public Access

CyberGIS-Cloud: A unified middleware framework for cloud-based geospatial research and education

Published: 08 July 2022 Publication History

Abstract

Interest in cloud-based cyberinfrastructure continues to grow within the geospatial community to tackle contemporary big data challenges. Distributed computing frameworks, deployed over the cloud, provide scalable and low-maintenance solutions to accelerate geospatial research and education. However, for scientists and researchers, the usage of such resources is highly constrained by the steep curve for learning diverse sets of platform-specific tools and APIs. This paper presents CyberGIS-Cloud as a unified middleware to streamline the execution of distributed geospatial workflows over multiple cloud backends with easy-to-use interfaces. CyberGIS-Cloud employs bringing computation-to-data model by abstracting and automating job execution over distributed resources hosted in the cloud environment where the data resides. We present details of CyberGIS-Cloud with support for popular distributed computing frameworks backed by research-oriented JetStream Cloud and commercial Google Cloud Platform.

References

[1]
2022. Production-Grade Container Orchestration. Retrieved March 19, 2022 from https://kubernetes.io
[2]
Ablimit Aji, Fusheng Wang, Hoang Vo, Rubao Lee, Qiaoling Liu, Xiaodong Zhang, and Joel Saltz. 2013. Hadoop GIS: A High Performance Spatial Data Warehousing System over Mapreduce. Proc. VLDB Endow. 6, 11 (Aug. 2013).
[3]
Furqan Baig, Hoang Vo, Tahsin Kurc, Joel Saltz, and Fusheng Wang. 2017. SparkGIS: Resource Aware Efficient In-Memory Spatial Query Processing. In Proceedings of the 25th ACM SIGSPATIAL. ACM.
[4]
Sanjay Dean, Jeffrey & Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM (2008).
[5]
Ahmed Eldawy. 2014. SpatialHadoop: Towards Flexible and Scalable Spatial Processing Using Mapreduce. In Proceedings of the 2014 SIGMOD PhD Symposium(SIGMOD’14 PhD Symposium). ACM, New York, NY, USA.
[6]
Ian Foster. 2011. Globus Online: Accelerating and Democratizing Science through Cloud-Based Services. IEEE Internet Computing 15, 3 (may 2011), 70–73. https://doi.org/10.1109/MIC.2011.64
[7]
Apache Software Foundation. 2022. Hadoop. Retrieved March 19, 2022 from https://hadoop.apache.org
[8]
Jinxuan Wu Jia Yu, Mohamed Sarwat. 2015. GeoSpark: A Cluster Computing Framework for Processing Large-Scale Spatial Data. In Proceedings of ACM SIGSPATIAL 2015.
[9]
Anand Padmanabhan, Ximo Ziao, Rebecca C. Vandewalle, Furqan Baig, Alexander Michel, Zhiyu Li, and Shaowen Wang. 2021. CyberGIS-Compute for Enabling Computationally Intensive Geospatial Research. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on APIs and Libraries for Geospatial Data Science (Beijing, China) (SpatialAPI ’21). Association for Computing Machinery, New York, NY, USA, Article 3, 2 pages. https://doi.org/10.1145/3486189.3490017
[10]
Yuxing Peng, Jonathan Skone, Callista Christ, and Hakizumwami Runesha. 2021. Skyway: A Seamless Solution for Bursting Workloads from On-Premises HPC Clusters to Commercial Clouds. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3437359.3465607
[11]
Matthew Rocklin. 2015. Dask: Parallel computation with blocked algorithms and task scheduling. In Proceedings of the 14th python in science conference. Citeseer.
[12]
Dong Xie, Feifei Li, Bin Yao, Gefei Li, Liang Zhou, and Minyi Guo. 2016. Simba: Efficient In-Memory Spatial Analytics. In In Proceedings of 35th ACM SIGMOD International Conference on Management of Data(SIGMOD’16).
[13]
Chaowei Yang, Robert Raskin, Michael Goodchild, and Mark Gahegan. 2010. Geospatial Cyberinfrastructure: Past, present and future. Computers, Environment and Urban Systems(2010).
[14]
Chaowei Yang, Manzhu Yu, Fei Hu, Yongyao Jiang, and Yun Li. 2017. Utilizing Cloud Computing to address big geospatial data challenges. Computers, Environment and Urban Systems(2017).
[15]
Dandong Yin, Yan Liu, Anand Padmanabhan, Jeff Terstriep, Johnathan Rush, and Shaowen Wang. 2017. A CyberGIS-Jupyter Framework for Geospatial Analytics at Scale. In Proceedings of the PEARC’17. ACM.
[16]
Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. 2016. Apache Spark: A Unified Engine for Big Data Processing. Commun. ACM (oct 2016).

Cited By

View all
  • (2024)Providing Accessible Software Environments Across Science Gateways and HPCPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670614(1-4)Online publication date: 17-Jul-2024
  • (2023)Cloud-Operated Open Literate Educational Resources: The Case of the MyBinderIEEE Transactions on Learning Technologies10.1109/TLT.2023.334369017(893-902)Online publication date: 19-Dec-2023
  • (2023)EasyScienceGateway: A new framework for providing reproducible user environments on science gatewaysConcurrency and Computation: Practice and Experience10.1002/cpe.792936:4Online publication date: 13-Oct-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PEARC '22: Practice and Experience in Advanced Research Computing 2022: Revolutionary: Computing, Connections, You
July 2022
455 pages
ISBN:9781450391610
DOI:10.1145/3491418
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cloud-computing
  2. cybergis
  3. middleware
  4. science-gateway

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

PEARC '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 133 of 202 submissions, 66%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)116
  • Downloads (Last 6 weeks)15
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Providing Accessible Software Environments Across Science Gateways and HPCPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670614(1-4)Online publication date: 17-Jul-2024
  • (2023)Cloud-Operated Open Literate Educational Resources: The Case of the MyBinderIEEE Transactions on Learning Technologies10.1109/TLT.2023.334369017(893-902)Online publication date: 19-Dec-2023
  • (2023)EasyScienceGateway: A new framework for providing reproducible user environments on science gatewaysConcurrency and Computation: Practice and Experience10.1002/cpe.792936:4Online publication date: 13-Oct-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media