skip to main content
10.1145/3626203.3670623acmconferencesArticle/Chapter ViewAbstractPublication PagespearcConference Proceedingsconference-collections
short-paper
Open access

Enhancing Research Productivity: Seamless Integration of Personal Devices and HPC Resources with the Cybershuttle Notebook Gateway

Published: 17 July 2024 Publication History

Abstract

Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network load and CPU usage across various computational workloads.

References

[1]
2018. CILogon. https://www.cilogon.org/.
[2]
2018. Jupyter Kernel Gateway. https://github.com/jupyter-server/kernel_gateway.
[3]
2024. Google Colab. https://colab.research.google.com.
[4]
2024. Jupyter Hub. https://jupyter.org/hub.
[5]
2024. Jupyter Lab. https://jupyterlab.readthedocs.io/.
[6]
J Eric Coulter, Eroma Abeysinghe, Sudhakar Pamidighantam, and Marlon Pierce. 2019. Virtual clusters in the jetstream cloud: A story of elasticized hpc. In Proceedings of the Humans in the Loop: Enabling and Facilitating Research on Cloud Computing. 1–6.
[7]
Dave Hudak, Doug Johnson, Alan Chalker, Jeremy Nicklas, Eric Franz, Trey Dockendorf, and Brian L McMichael. 2018. Open OnDemand: A web-based client portal for HPC centers. Journal of Open Source Software 3, 25 (2018), 622.
[8]
Suresh Marru, Lahiru Gunathilake, Chathura Herath, Patanachai Tangchaisin, Marlon Pierce, Chris Mattmann, Raminder Singh, Thilina Gunarathne, Eran Chinthaka, Ross Gardler, 2011. Apache airavata: a framework for distributed applications and computational workflows. In Proceedings of the 2011 ACM workshop on Gateway computing environments. 21–28.
[9]
Suresh Marru, Marlon Pierce, Beth Plale, Sudhakar Pamidighantam, Dimuthu Wannipurage, Marcus Christie, Isuru Ranawaka, Eroma Abeysinghe, Rob Quick, Emad Tajkhorshid, 2023. Cybershuttle: An End-to-End Cyberinfrastructure Continuum to Accelerate Discovery in Science and Engineering. In Practice and Experience in Advanced Research Computing. 26–34.
[10]
Drew Paine and Lavanya Ramakrishnan. 2022. Understanding interactive and reproducible computing with jupyter tools at facilities. (2022).
[11]
João Felipe Pimentel, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. A large-scale study about quality and reproducibility of jupyter notebooks. In International Conference on Mining Software Repositories (MSR). IEEE, 507–517.
[12]
Isuru Ranawaka, Suresh Marru, Juleen Graham, Aarushi Bisht, Jim Basney, Terry Fleury, Jeff Gaynor, Dimuthu Wannipurage, Marcus Christie, Alexandru Mahmoud, Enis Afgan, and Marlon Pierce. 2020. Custos: Security Middleware for Science Gateways(PEARC ’20). Association for Computing Machinery, New York, NY, USA, 278–284.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PEARC '24: Practice and Experience in Advanced Research Computing 2024: Human Powered Computing
July 2024
608 pages
ISBN:9798400704192
DOI:10.1145/3626203
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2024

Check for updates

Author Tags

  1. HPC
  2. JupyterLab

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Funding Sources

Conference

PEARC '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 133 of 202 submissions, 66%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 144
    Total Downloads
  • Downloads (Last 12 months)144
  • Downloads (Last 6 weeks)41
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media