skip to main content
10.1145/3311790.3396657acmconferencesArticle/Chapter ViewAbstractPublication PagespearcConference Proceedingsconference-collections
research-article

JetLag: An Interactive, Asynchronous Array Computing Environment

Published: 26 July 2020 Publication History

Abstract

We describe an interactive computing environment called JetLag. JetLag implements the following features of Phylanx project: (1) Phylanx, a Python-based asynchronous array computing toolkit; (2) the APEX performance measurement library; (3) a performance visualization framework called Traveler; (4) the Tapis/Agave Science as a Service middleware; and (6) a container infrastructure that includes Docker-based Jupyter notebook for the client and a singularity image for the server.
The running system starts with a user performing array computations on their workstation or laptop. If, at some point, the calculation the user is performing becomes sufficiently intensive or numerous, it can be packaged and sent to another machine where it will run (through the batch queue system if there is one), produce a result, and have that result sent back to the user’s local interface. Whether the calculation is local or remote, the user will be able to use APEX and Traveler to diagnose and fix performance related problems.
The JetLag system is suitable for a variety of array computational tasks, including machine learning and exploratory data analysis.

Supplemental Material

MP4 File
Presentation video

References

[1]
C Bishop. 2007. Pattern Recognition and Machine Learning (Information Science and Statistics), 1st edn. 2006. corr. 2nd printing edn. Springer, New York (2007).
[2]
Rion Dooley, Steven R Brandt, and John Fonner. 2018. The Agave Platform: An Open, Science-as-a-Service Platform for Digital Science. In Proceedings of the Practice and Experience on Advanced Research Computing. ACM, 28.
[3]
Dominic Eschweiler, Michael Wagner, Markus Geimer, Andreas Knüpfer, Wolfgang E Nagel, and Felix Wolf. 2012. Open Trace Format 2: The Next Generation of Scalable Trace Formats and Support Libraries.In Advances in Parallel Computing. Vol. 22. IOS Press, Amsterdam, NL, 481–490.
[4]
F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4(2015), 1–19.
[5]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on. Ieee, 263–272.
[6]
Kevin Huck, Allan Porterfield, Nick Chaimov, Hartmut Kaiser, Allen Malony, Thomas Sterling, and Rob Fowler. 2015. An Autonomic Performance Environment for Exascale. Supercomputing Frontiers and Innovations 2, 3 (2015), 49–66. https://superfri.org/superfri/article/view/64
[7]
Klaus Iglberger, Georg Hager, Jan Treibig, and Ulrich Rüde. 2012. High performance smart expression template math libraries. In 2012 International Conference on High Performance Computing & Simulation (HPCS). IEEE, 367–373.
[8]
Hartmut Kaiser, Thomas Heller, Bryce Adelstein-Lelbach, Adrian Serio, and Dietmar Fey. 2014. Hpx: A task based programming model in a global address space. In Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models. ACM, 6.
[9]
Dominic C Marcello, Kundan Kadam, Geoffrey C Clayton, Juhan Frank, Hartmut Kaiser, and Patrick M Motl. 2016. Introducing Octo-tiger/HPX: Simulating interacting binaries with adaptive mesh refinement and the fast multipole method. Proceedings of Science(2016), 13–17.
[10]
Cristian Tapus, I-Hsin Chung, and Jeffrey K Hollingsworth. 2002. Active harmony: Towards automated performance tuning. In SC’02: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing. IEEE, ACM/IEEE, Baltimore, Maryland, USA, 44–44.
[11]
Dan Terpstra, Heike Jagode, Haihang You, and Jack Dongarra. 2010. Collecting performance data with PAPI-C. In Tools for High Performance Computing 2009. Springer, 157–173.
[12]
R Tohid, Bibek Wagle, Shahrzad Shirzad, Patrick Diehl, Adrian Serio, Alireza Kheirkhahan, Parsa Amini, Katy Williams, Kate Isaacs, Kevin Huck, 2018. Asynchronous Execution of Python Code on Task-Based Runtime Systems. In 2018 IEEE/ACM 4th International Workshop on Extreme Scale Programming Models and Middleware (ESPM2). IEEE, 37–45.
[13]
Bibek Wagle, Mohammad Alaul Haque Monil, Kevin Huck, Allen D. Malony, Adrian Serio, and Hartmut Kaiser. 2019. Runtime Adaptive Task Inlining on Asynchronous Multitasking Runtime Systems. In Proceedings of the 48th International Conference on Parallel Processing (Kyoto, Japan) (ICPP 2019). Association for Computing Machinery, New York, NY, USA, Article Article 76, 10 pages. https://doi.org/10.1145/3337821.3337915
[14]
Katy Williams, Alex Bigelow, and Katherine E Issacs. 2020. Visualizing a Moving Target: A Design Study on Task Parallel Programs in the Presence of Evolving Data and Concerns. IEEE Transactions on Visualization and Computer Graphics 26, 1 (Jan 2020), 1118–1128. https://doi.org/10.1109/TVCG.2019.2934285

Cited By

View all
  • (2022)Halide Code Generation Framework in PhylanxEuro-Par 2022: Parallel Processing Workshops10.1007/978-3-031-31209-0_3(32-45)Online publication date: 22-Aug-2022
  • (2021)An interface for multidimensional arrays in Arkouda2021 IEEE High Performance Extreme Computing Conference (HPEC)10.1109/HPEC49654.2021.9622858(1-2)Online publication date: 20-Sep-2021
  • (2020)Distributed Asynchronous Array Computing with the JetLag Environment2020 IEEE/ACM 9th Workshop on Python for High-Performance and Scientific Computing (PyHPC)10.1109/PyHPC51966.2020.00011(49-57)Online publication date: Nov-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PEARC '20: Practice and Experience in Advanced Research Computing 2020: Catch the Wave
July 2020
556 pages
ISBN:9781450366892
DOI:10.1145/3311790
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. array
  2. asynchronous
  3. cloud computing
  4. interactive computing
  5. performance tuning
  6. performance visualization
  7. research environment

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

PEARC '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 133 of 202 submissions, 66%

Upcoming Conference

PEARC '25
Practice and Experience in Advanced Research Computing
July 20 - 24, 2025
Columbus , OH , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Halide Code Generation Framework in PhylanxEuro-Par 2022: Parallel Processing Workshops10.1007/978-3-031-31209-0_3(32-45)Online publication date: 22-Aug-2022
  • (2021)An interface for multidimensional arrays in Arkouda2021 IEEE High Performance Extreme Computing Conference (HPEC)10.1109/HPEC49654.2021.9622858(1-2)Online publication date: 20-Sep-2021
  • (2020)Distributed Asynchronous Array Computing with the JetLag Environment2020 IEEE/ACM 9th Workshop on Python for High-Performance and Scientific Computing (PyHPC)10.1109/PyHPC51966.2020.00011(49-57)Online publication date: Nov-2020
  • (2020)Towards a Scalable and Distributed Infrastructure for Deep Learning Applications2020 IEEE/ACM Fourth Workshop on Deep Learning on Supercomputers (DLS)10.1109/DLS51937.2020.00008(20-30)Online publication date: Nov-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media