Experiences Migrating the Agave Platform to A Kubernetes Native System on the Jetstream Cloud
Article No.: 39, Pages 1 - 4
Abstract
The Agave Platform is a Science-as-a-Service (ScaaS) platform for reproducible science. The current production deployments of the platform are deployed and managed using Ansible and Docker Compose. While capable, this has historically led to operational complexity for those adopting the platform. Over the last year, we have worked to migrate the platform to a Kubernetes native deployment. In this paper we discuss our experiences evolving the platform, its architecture, and getting the most out of the Jetstream cloud.
References
[1]
beanstalkd [n.d.]. Beanstalkd. https://beanstalkd.github.io/
[2]
bitnamihelm [n.d.]. Helm Charts to deploy MariaDB in Kubernetes. https://bitnami.com/stack/mariadb/helm
[3]
Rion Dooley. 2021. Agave Helm Charts. https://github.com/agaveplatform/helm-charts
[4]
Rion Dooley and Steven R. Brandt. 2020. From Zero to Science-as-a-Service with the Agave Platform, Helm, and Kubernetes. https://osf.io/4nrd5/
[5]
elasticstack [n.d.]. ELK Stack: Elasticsearch, Logstash, Kibana. https://www.elastic.co/what-is/elk-stack
[6]
Ansible Hat, Red. [n.d.]. Ansible is Simple IT Automation. https://www.ansible.com
[7]
hpa [n.d.]. Horizontal Pod Autoscaler. https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/Section: docs.
[8]
AppsCode Inc.[n.d.]. KubeDB by AppsCode. https://kubedb.com
[9]
istio [n.d.]. Istio Servish Mesh. https://istio.io/
[10]
kubespray [n.d.]. kubernetes-sigs/kubespray. https://github.com/kubernetes-sigs/kubespray
[11]
linkerd [n.d.]. Linkerd: The world’s lightest, fastest service mesh.https://linkerd.io/
[12]
loki 2021. Loki. https://github.com/grafana/lokioriginal-date: 2018-04-16T09:22:48Z.
[13]
maesh [n.d.]. Traefik Mesh, the Simplest Service Mesh | Traefik Labs. https://traefik.io/traefik-mesh/
[14]
makdharma. 2017. gRPC Load Balancing. https://grpc.io/blog/grpc-load-balancing/ Section: blog.
[15]
manilla [n.d.]. Manila - OpenStack. https://wiki.openstack.org/wiki/Manila
[16]
William Morgan. 2018. gRPC Load Balancing on Kubernetes without Tears. https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears/ Section: blog.
[17]
nats [n.d.]. NATS - Open Source Messaging System. https://nats.io/
[18]
perconahelmcharts 2021. Percona Helm Charts. https://github.com/percona/percona-helm-chartsoriginal-date: 2019-07-10T18:31:41Z.
[19]
persistentvolumes [n.d.]. Persistent Volumes. https://kubernetes.io/docs/concepts/storage/persistent-volumes/Section: docs.
[20]
Prometheus. [n.d.]. Prometheus - Monitoring system & time series database. https://prometheus.io/
[21]
Rion Dooley, Steven R. Brandt, and John Fonner. 2018. The Agave Platform: An Open, Science-as-a-Service Platform for Digital Science. In Proceedings of the 2018 Practice & Experience in Advanced Research Computing. ACM, Pittsburgh, PA. https://doi.org/10.1145/3219104.3219129
[22]
Sean B. Cleveland, Rion Dooley, David Perry, Joe Stubbs, John M. Fonner, and Gwen A. Jacobs. 2018. Building Science Gateway Infrastructure in the Middle of the Pacific: Experiences using the Agave Deployer and Agave Platform to build science gateways. In Proceedings of the 2018 Practice & Experience in Advanced Research Computing. ACM, Pittsburgh, PA. https://doi.org/10.1145/3219104.3219151
[23]
Craig A. Stewart, George Turner, Matthew Vaughn, Niall I. Gaffney, Timothy M. Cockerill, Ian Foster, David Hancock, Nirav Merchant, Edwin Skidmore, Daniel Stanzione, James Taylor, and Steven Tuecke. 2015. Jetstream: a self-provisioned, scalable science and engineering cloud environment. In Proceedings of the 2015 XSEDE Conference on Scientific Advancements Enabled by Enhanced Cyberinfrastructure - XSEDE ’15. ACM Press, St. Louis, Missouri, 1–8. https://doi.org/10.1145/2792745.2792774
[24]
traefik [n.d.]. Traefik, The Cloud Native Application Proxy. https://traefik.io/traefik/
[25]
volumesnapshots [n.d.]. Volume Snapshots. https://kubernetes.io/docs/concepts/storage/volume-snapshots/Section: docs.
Recommendations
The Agave Platform: An Open, Science-as-a-Service Platform for Digital Science
PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing: Seamless CreativityThe Agave Platform first appeared in 2011 as a pilot project for the iPlant Collaborative [11]. In its first two years, Foundation saw over 40% growth per month, supporting 1000+ clients, 600+ applications, 4 HPC systems at 3 centers across the US. It ...
Smuggling Multi-cloud Support into Cloud-native Applications using Elastic Container Platforms
CLOSER 2017: Proceedings of the 7th International Conference on Cloud Computing and Services ScienceElastic container platforms (like Kubernetes, Docker Swarm, Apache Mesos) fit very well with existing cloud-native application architecture approaches. So it is more than astonishing, that these already existing and open source available elastic ...
An efficient multi-task PaaS cloud infrastructure based on docker and AWS ECS for application deployment
The setup environment and deployment of distributed applications is a human intensive and highly complex process that poses significant challenges. Nowadays many applications are developed in the cloud and existing applications are migrated to the cloud ...
Comments
Information & Contributors
Information
Published In
July 2021
310 pages
ISBN:9781450382922
DOI:10.1145/3437359
Copyright © 2021 ACM.
Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 July 2021
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
- Nation Science Foundation
Conference
Acceptance Rates
Overall Acceptance Rate 133 of 202 submissions, 66%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 87Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)2
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format