- Sponsor:
- sigsoft online
- sigmetrics
ICPE'22 is in the past, and for the first time the conference's companion proceedings are published in form of post-conference proceedings. The main motivation of this was to give authors of workshop or short papers an opportunity to improve their archived research papers based on discussions during the conference. This post-proceedings collect material for the following tracks:
Work-in-Progress and Vision Track: The work-in-progress and vision track this year was organized by Cristina L. Abad. The goal of this track was for attendees to present, and get feedback on, early ideas. Two papers were accepted in this track.
Poster and Demonstrations Track: Christoph Laaber and Wen Xia headed the poster and demonstrations track. Four papers were accepted and presented in a special session on the first conference day.
Tutorials: Under the leadership of David Daly and Shuibing He, three high-quality tutorials were organized at the conference this year: - "Optimizing the Performance of Fog Computing Environments Using AI and Co-Simulation", by Shreshth Tuli and Giuliano Casale - "Automated Benchmarking of cloud-hosted DBMS with benchANT", by Daniel Seybold and Jörg Domaschka - "SPEC Server Efficiency Benchmark Development - How to Contribute to the Future of Energy Conservation", by Maximilian Meissner, Klaus-Dieter Lange, Jeremy Arnold, Sanjay Sharma, Roger Tipley, Nishant Rawtani, David Reiner, Mike Petrich, Aaron Cragin
Data Challenge Track: The first data challenge track ever at ICPE was organized by Cor-Paul Bezemer (University of Alberta), David Daly (MongoDB) and Weiyi Shang (Concordia University), with the support of 5 PC members. In this track, an industrial performance dataset was provided by MongoDB. The participants were invited to come up with research questions about the dataset, and study those. The challenge was open-ended: participants can choose the research questions that they find most interesting. The data challenge track accepted 4 short papers, in which the proposed approaches and/or tools and their findings are discussed.
Proceeding Downloads
FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles: Vision Paper
In the last decades, especially intensified by the pandemic situation in which many people stay at home and order goods online, the need for efficient logistics systems has increased significantly. Hence, the performance of optimization techniques for ...
B-MEG: Bottlenecked-Microservices Extraction Using Graph Neural Networks
The microservices architecture enables independent development and maintenance of application components through its fine-grained and modular design. This has enabled rapid adoption of microservices architecture to build latency-sensitive online ...
SPEC Research - Introducing the Predictive Data Analytics Working Group: Poster Paper
- André Bauer,
- Mark Leznik,
- Md Shahriar Iqbal,
- Daniel Seybold,
- Igor Trubin,
- Benjamin Erb,
- Jörg Domaschka,
- Pooyan Jamshidi
The research field of data analytics has grown significantly with the increase of gathered and available data. Accordingly, a large number of tools, metrics, and best practices have been proposed to make sense of this vast amount of data. To this end, ...
SPEChpc 2021 Benchmark Suites for Modern HPC Systems
- Junjie Li,
- Alexander Bobyr,
- Swen Boehm,
- William Brantley,
- Holger Brunst,
- Aurelien Cavelan,
- Sunita Chandrasekaran,
- Jimmy Cheng,
- Florina M. Ciorba,
- Mathew Colgrove,
- Tony Curtis,
- Christopher Daley,
- Mauricio Ferrato,
- Mayara Gimenes de Souza,
- Nick Hagerty,
- Robert Henschel,
- Guido Juckeland,
- Jeffrey Kelling,
- Kelvin Li,
- Ron Lieberman,
- Kevin McMahon,
- Egor Melnichenko,
- Mohamed Ayoub Neggaz,
- Hiroshi Ono,
- Carl Ponder,
- Dave Raddatz,
- Severin Schueller,
- Robert Searles,
- Fedor Vasilev,
- Veronica Melesse Vergara,
- Bo Wang,
- Bert Wesarg,
- Sandra Wienke,
- Miguel Zavala
The SPEChpc 2021 suites are application-based benchmarks de- signed to measure performance of modern HPC systems. The bench- marks support MPI, MPI+OpenMP, MPI+OpenMP target offload, MPI+OpenACC and are portable across all major HPC platforms.
MAPLE: Model Aggregation and Prediction for Learned Ecosystem
Many Artificial Intelligence (AI) applications are composed of multiple machine learning (ML) and deep learning (DL) models. Intelligent process automation (IPA) requires a combination (sequential or parallel) of models to complete an inference task. ...
HLS_Profiler: Non-Intrusive Profiling tool for HLS based Applications
The High-Level Synthesis (HLS) tools aid in simplified and faster design development without familiarity with Hardware Description Language (HDL) and Register Transfer Logic (RTL) design flow that can be implemented on an FPGA (Field Programmable Gate ...
SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation
- Maximilian Meissner,
- Klaus-Dieter Lange,
- Jeremy Arnold,
- Sanjay Sharma,
- Roger Tipley,
- Nishant Rawtani,
- David Reiner,
- Mike Petrich,
- Aaron Cragin
A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers ...
Optimizing the Performance of Fog Computing Environments Using AI and Co-Simulation
This tutorial presents a performance engineering approach for optimizing the Quality of Service (QoS) of Edge/Fog/Cloud Computing environments using AI and Coupled-Simulation being developed as part of the Co-Simulation based Container Orchestration (...
Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning: Data Challenge Paper
Performance regression testing is a foundation of modern DevOps processes and pipelines. Thus, the detection of change points, i.e., updates or commits that cause a significant change in the performance of the software, is of special importance. ...
Characterizing and Triaging Change Points
Testing software performance continuously can greatly benefit from automated verification done on continuous integration (CI) servers, but it generates a large number of performance test data with noise. To identify the change points in test data, ...
Beware of the Interactions of Variability Layers When Reasoning about Evolution of MongoDB
With commits and releases, hundreds of tests are run on varying conditions (e.g., over different hardware and workload) that can help to understand evolution and ensure non-regression of software performance. We hypothesize that performance is not only ...
Change Point Detection for MongoDB Time Series Performance Regression
Commits to the MongoDB software repository trigger a collection of automatically run tests. Here, the identification of commits responsible for performance regressions is paramount. Previously, the process relied on manual inspection of time series ...
FaaSET: A Jupyter Notebook to Streamline Every Facet of Serverless Development
Function-as-a-Service platforms require developers to use many different tools and services for function development, packaging, deployment, debugging, testing, orchestration of experiments, and analysis of results. Diverse toolchains are necessary due ...
Beauty and the Beast: A Case Study on Performance Prototyping of Data-Intensive Containerized Cloud Applications
Data-intensive container-based cloud applications have become popular with the increased use cases in the Internet of Things domain. Challenges arise when engineering such applications to meet quality requirements, both classical ones like performance ...
Measuring Baseline Overheads in Different Orchestration Mechanisms for Large FaaS Workflows
Serverless environments have attracted significant attention in recent years as a result of their agility in execution as well as inherent scaling capabilities as a cloud-native execution model. While extensive analysis has been performed in various ...
Characterizing X86 and ARM Serverless Performance Variation: A Natural Language Processing Case Study
In this paper, we leverage a Natural Language Processing (NLP) pipeline for topic modeling consisting of three functions for data preprocessing, model training, and inferencing to analyze serverless platform performance variation. Specifically, we ...
MiSeRTrace: Kernel-level Request Tracing for Microservice Visibility
- Thrivikraman V,
- Vishnu R. Dixit,
- Nikhil Ram S,
- Vikas K. Gowda,
- Santhosh Kumar Vasudevan,
- Subramaniam Kalambur
With the evolution of microservice applications, the underlying architectures have become increasingly complex compared to their monolith counterparts. This mainly brings in the challenge of observability. By providing a deeper understanding into the ...
TaskFlow: An Energy- and Makespan-Aware Task Placement Policy for Workflow Scheduling through Delay Management
Datacenters need to become more power efficient for political and climate reasons. In this work, we introduce an idea for the community to further explore. We embed the idea in TaskFlow: a makespan conservative, energy-aware task placement policy for ...
CTT: Load Test Automation for TOSCA-based Cloud Applications
Despite today's fast and rapid modeling and deployment capabilities to meet customer requirements in an agile manner, testing is still of utmost importance to avoid outages, unsatisfied customers, and performance problems. To tackle such issues, (load) ...
Benchmarking Runtime Scripting Performance in Wasmer
In this paper, we explore the use of Wasmer and WebAssembly (WASM) as a sandboxed environment for general-purpose runtime scripting. Our work differs from prior research focusing on browser-based performance or SPEC benchmarks. In particular, we use ...
How is Transient Behavior Addressed in Practice?: Insights from a Series of Expert Interviews
Transient behavior occurs when a running software system changes from one steady-state to another. In microservice systems, such disruptions can, for example, be caused by continuous deployment, self-adaptation, and various failures. Although transient ...
Design-time Performability Optimization of Runtime Adaptation Strategies
Self-Adaptive Systems (SASs) adapt themselves to environmental changes during runtime to maintain Quality of Service (QoS) goals. Designing and optimizing the adaptation strategy of an SAS regarding its impact on quality properties is a challenging ...
Analysis of Garbage Collection Patterns to Extend Microbenchmarks for Big Data Workloads
Java uses automatic memory allocation where the user does not have to explicitly free used memory. This is done by the garbage collector. Garbage Collection (GC) can take up a significant amount of time, especially in Big Data applications running large ...
A Multiserver Approximation for Cloud Scaling Analysis
Queueing models of web service systems run at increasingly large scales, with large customer populations and with multiservers introduced by scaling up the services. "Scalable" multiserver approximations, in the sense that they that are insensitive to ...
Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency
Energy efficiency has become a major concern in the IT sector as the energy demand for data centers is projected to reach 1PWh per year by 2030. While hardware designers improve the energy efficiency of their products, software developers often do not ...
Performance Evaluation of GraphCore IPU-M2000 Accelerator for Text Detection Application
The large compute load and memory footprint of modern deep neural networks motivates the use of accelerators for high through- put deployments in application spanning multiple domains. In this paper, we evaluate throughput capabilities of a ...
Index Terms
- Companion of the 2022 ACM/SPEC International Conference on Performance Engineering