short-paper

Public Access

Consecutive Job Submission Behavior at Mira Supercomputer

Authors:

Stephan Schlagkamp,

Rafael Ferreira da Silva,

William Allcock,

Uwe SchwiegelshohnAuthors Info & Claims

HPDC '16: Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing

Pages 93 - 96

https://doi.org/10.1145/2907294.2907314

Published: 31 May 2016 Publication History

Abstract

Understanding user behavior is crucial for the evaluation of scheduling and allocation performances in HPC environments. This paper aims to further understand the dynamic user reaction to different levels of system performance by performing a comprehensive analysis of user behavior in recorded data in the form of delays in the subsequent job submission behavior. Therefore, we characterize a workload trace covering one year of job submissions from the Mira supercomputer at ALCF (Argonne Leadership Computing Facility). We perform an in-depth analysis of correlations between job characteristics, system performance metrics, and the subsequent user behavior. Analysis results show that the user behavior is significantly influenced by long waiting times, and that complex jobs (number of nodes and CPU hours) lead to longer delays in subsequent job submissions.

References

[1]

P. Carns et al. Understanding and improving computational science storage access through continuous characterization. ACM TOS, 7(3), 2011.

Digital Library

[2]

D. G. Feitelson. Looking at data. In IEEE IPDPS, 2008.

[3]

D. G. Feitelson. Workload modeling for computer systems performance evaluation. Cambridge University Press, 2015.

Digital Library

[4]

D. G. Feitelson and A. W. Mu'alem. On the definition of on-line in job scheduling problems. ACM SIGACT News, 36(1), 2005.

Digital Library

[5]

D. G. Feitelson and E. Shmueli. A case for conservative workload modeling: Parallel job scheduling with daily cycles of activity. In IEEE MASCOTS, 2009.

[6]

R. Ferreira da Silva et al. Characterizing a high throughput computing workload: The compact muon solenoid (CMS) experiment at LHC. Procedia Computer Science, 51, 2015.

[7]

R. Ferreira da Silva and T. Glatard. A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions. In Euro-Par. Springer, 2013.

Digital Library

[8]

A. Geist and D. A. Reed. A survey of high-perfor- mance computing scaling challenges. IJHPCA, 2015.

[9]

D. L. Hart. Measuring TeraGrid: workload characterization for a high-performance computing federation. IJHPCA, 25(4), 2011.

Digital Library

[10]

C. B. Lee and A. Snavely. On the user--scheduler dialogue: studies of user-provided runtime estimates and utility functions. IJHPCA, 20(4), 2006.

Digital Library

[11]

H. Luu et al. A multiplatform study of I/O behavior on petascale supercomputers. In ACM HPDC, 2015.

Digital Library

[12]

D. A. Reed and J. Dongarra. Exascale computing and big data. Communications of the ACM, 58(7), 2015.

Digital Library

[13]

J. Renker et al. Questionnaire for user habits of compute clusters (QUHCC). In HCI International. Springer, 2015.

[14]

G. P. Rodrigo Álvarez et al. HPC system lifetime story: Workload characterization and evolutionary analyses on NERSC systems. In ACM HPDC, 2015.

Digital Library

[15]

S. Schlagkamp. Influence of dynamic think times on parallel job scheduler performances in generative simulations. In JSSPP, 2015.

[16]

S. Schlagkamp and J. Renker. Acceptance of waiting times in high performance computing. In HCI International. Springer, 2015.

[17]

U. Schwiegelshohn. How to design a job scheduling algorithm. In JSSPP. Springer, 2014.

[18]

E. Shmueli and D. G. Feitelson. Using site-level modeling to evaluate the performance of parallel system schedulers. In IEEE MASCOTS, 2006.

Digital Library

[19]

E. Shmueli and D. G. Feitelson. On simulation and design of parallel-systems schedulers: are we doing the right thing? IEEE TPDS, 20(7), 2009.

Digital Library

[20]

N. Zakay and D. G. Feitelson. On identifying user session boundaries in parallel workload logs. In JSSPP. Springer, 2013.

Cited By

Yanhaona MGrimshaw AMickey S(2024)HighP5: Programming using Partitioned Parallel Processing SpacesJournal of the Brazilian Computer Society10.5753/jbcs.2024.434530:1(653-687)Online publication date: 17-Dec-2024
https://doi.org/10.5753/jbcs.2024.4345
Du YMarchal LPallez GRobert Y(2024)Improving batch schedulers with node stealing for failed jobsConcurrency and Computation: Practice and Experience10.1002/cpe.804336:12Online publication date: 16-Feb-2024
https://doi.org/10.1002/cpe.8043
Jakobsche TLachiche NCiorba F(2023)Investigating HPC Job Resource Requests and Job Efficiency Reporting2023 22nd International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC59212.2023.00024(61-68)Online publication date: Jul-2023
https://doi.org/10.1109/ISPDC59212.2023.00024
Show More Cited By

Index Terms

Consecutive Job Submission Behavior at Mira Supercomputer
1. Computing methodologies
  1. Modeling and simulation

Recommendations

User-level Workload Analysis for Supercomputers
ICSIM '21: Proceedings of the 2021 4th International Conference on Software Engineering and Information Management

The analysis of workloads is important for understanding how supercomputing systems are used. Gaining insight into user behavior is equally crucial for current system in aspects of job scheduling, promoting allocation efficiency and improving user ...
Job submission to grid computing environments
UK e-Science All Hands Meeting 2006

The problem of enabling scientist users to submit jobs to grid computing environments will eventually limit the usability of grids. The eMinerals project has tackled this problem by developing the 'my_condor_submit' (MCS) tool, which provides a simple ...
Understanding User Behavior

In this paper, we investigate the differences and similarities in user job submission behavior in High Performance Computing (HPC) and High Throughput Computing (HTC). We consider job submission behavior in terms of parallel batch-wise submissions, as ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HPDC '16: Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing

May 2016

302 pages

ISBN:9781450343145

DOI:10.1145/2907294

General Chair:
Hiroshi Nakashima
Kyoto University, Japan
,
Program Chairs:
Kenjiro Taura
The University of Tokyo, Japan
,
Jack Lange
University of Pittsburgh, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

University of Arizona: University of Arizona
SIGARCH: ACM Special Interest Group on Computer Architecture

In-Cooperation

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

DOE
German Research Foundation

Conference

HPDC'16

Sponsor:

University of Arizona
SIGARCH

HPDC'16: The 25th International Symposium on High-Performance Parallel and Distributed Computing

May 31 - June 4, 2016

Kyoto, Japan

Acceptance Rates

HPDC '16 Paper Acceptance Rate 20 of 129 submissions, 16%;

Overall Acceptance Rate 166 of 966 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
290
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)8

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yanhaona MGrimshaw AMickey S(2024)HighP5: Programming using Partitioned Parallel Processing SpacesJournal of the Brazilian Computer Society10.5753/jbcs.2024.434530:1(653-687)Online publication date: 17-Dec-2024
https://doi.org/10.5753/jbcs.2024.4345
Du YMarchal LPallez GRobert Y(2024)Improving batch schedulers with node stealing for failed jobsConcurrency and Computation: Practice and Experience10.1002/cpe.804336:12Online publication date: 16-Feb-2024
https://doi.org/10.1002/cpe.8043
Jakobsche TLachiche NCiorba F(2023)Investigating HPC Job Resource Requests and Job Efficiency Reporting2023 22nd International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC59212.2023.00024(61-68)Online publication date: Jul-2023
https://doi.org/10.1109/ISPDC59212.2023.00024
Li BArora RSamsi SPatel TArcand WBestor DByun CRoy RBergeron BHolodnak JHoule MHubbell MJones MKepner JKlein AMichaleas PMcDonald JMilechin LMullen JProut APrice BReuther ARosa AWeiss MYee CEdelman DVanterpool ACheng AGadepally VTiwari D(2022)AI-Enabling Workloads on Large-Scale GPU-Accelerated System: Characterization, Opportunities, and Implications2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA)10.1109/HPCA53966.2022.00093(1224-1237)Online publication date: Apr-2022
https://doi.org/10.1109/HPCA53966.2022.00093
Maheshwari KWilkinson SMay ASkluzacek TKuchar OFerreira da Silva R(2022)Pseudonymization at Scale: OLCF’s Summit Usage Data Case Study2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020380(3432-3440)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020380
Wang QShen YLi J(2021)User-level Workload Analysis for SupercomputersProceedings of the 2021 4th International Conference on Software Engineering and Information Management10.1145/3451471.3451483(68-73)Online publication date: 16-Jan-2021
https://dl.acm.org/doi/10.1145/3451471.3451483
Patel TLiu ZKettimuthu RRich PAllcock WTiwari DCuicchi CQualters IKramer W(2020)Job characteristics on large-scale systemsProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.5555/3433701.3433812(1-17)Online publication date: 9-Nov-2020
https://dl.acm.org/doi/10.5555/3433701.3433812
Patel TLiu ZKettimuthu RRich PAllcock WTiwari D(2020)Job Characteristics on Large-Scale Systems: Long-Term Analysis, Quantification, and ImplicationsSC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00088(1-17)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00088
Yu JLiu GDong WLi X(2019)WatCacheThe Journal of Supercomputing10.1007/s11227-017-2167-775:2(554-586)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s11227-017-2167-7
Yu JLiu GLiu XDong WLi XLiu Y(2018)Rethinking Node Allocation Strategy for Data-intensive Applications in Consideration of Spatially Bursty I/OProceedings of the 2018 International Conference on Supercomputing10.1145/3205289.3205305(12-21)Online publication date: 12-Jun-2018
https://dl.acm.org/doi/10.1145/3205289.3205305
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten