research-article

Free access

Hybrid Machine Learning/Analytical Models for Performance Prediction: A Tutorial

Authors:

Paolo RomanoAuthors Info & Claims

ICPE '15: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

Pages 341 - 344

https://doi.org/10.1145/2668930.2688823

Published: 31 January 2015 Publication History

Abstract

Classical approaches to performance prediction of computer systems rely on two, typically antithetic, techniques: Machine Learning (ML) and Analytical Modeling (AM).

ML undertakes a black-box approach, which typically achieves very good accuracy in regions of the features' space that have been sufficiently explored during the training process, but that has very weak extrapolation power (i.e., poor accuracy in regions for which none, or too few samples are known).

Conversely, AM relies on a white-box approach, whose key advantage is that it requires no or minimal training, hence supporting prompt instantiation of the target system's performance model. However, to ensure their tractability, AM-based performance predictors typically rely on simplifying assumptions. Consequently, AM's accuracy is challenged in scenarios not matching such assumptions.

This tutorial describes techniques that exploit AM and ML in synergy in order to get the best of the two worlds. It surveys several such hybrid techniques and presents use cases spanning a wide range of application domains.

References

[1]

C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). 2007.

Digital Library

[2]

J. Chen et al. Model ensemble tools for self-management in data centers. In Proc. of ICDE Workshops, 2013.

[3]

P. Di Sanzo et al. A flexible framework for accurate simulation of cloud in-memory data stores. ArXiv e-prints, Dec. 2014.

[4]

D. Didona et al. Identifying the optimal level of parallelism in transactional memory applications. Springer Computing Journal, 2013.

Digital Library

[5]

D. Didona et al. Transactional auto scaler: Elastic scaling of replicated in-memory transactional data grids. ACM Trans. Auton. Adapt. Syst., 9(2):11:1-11:32, July 2014.

Digital Library

[6]

D. Didona et al. Enhancing Performance Prediction Robustness by Combining Analytical Modeling and Machine Learning. In Proc. of ICPE, 2015.

Digital Library

[7]

D. Didona and P. Romano. On Bootstrapping Machine Learning Performance Predictors via Analytical Models. ArXiv e-prints, Oct. 2014.

[8]

D. Didona and P. Romano. Performance modelling of partially replicated in-memory transactional stores. In Proc. of MASCOTS, 2014.

Digital Library

[9]

P. Romano and M. Leonetti. Self-tuning batching in total order broadcast protocols via analytical modelling and reinforcement learning. In Proc. of ICNC, 2011.

[10]

D. Rughetti et al. Analytical/ml mixed approach for concurrency regulation in software transactional memory. In Proc. of CCGRID, 2014.

[11]

Y. C. Tay. Analytical Performance Modeling for Computer Systems. Morgan & Claypool Publishers, 2013.

Digital Library

[12]

L. Kleinrock Queueing Systems, Theory, Volume 1. Wiley Interscience, 1975.

Digital Library

[13]

G. Tesauro et al. On the use of hybrid reinforcement learning for autonomic resource allocation. Cluster Computing, 2007.

Digital Library

[14]

E. Thereska and G. Ganger. Ironmodel: Robust performance models in the wild. In Proc. of SIGMETRICS, 2008.

Digital Library

[15]

M. Hall et al. The WEKA Data Mining Software: An Update. SIGKDD Explor. Newsl., 11(1) 10{18, June 2009.

Digital Library

[16]

M. Couceiro et al. A machine learning approach to performance prediction of total order broadcast protocols. In Proc. of SASO, 2010.

Digital Library

[17]

P. Bernstein and E. Newcomer Principles of Transaction Processing: For the Systems Professional Morgan Kaufmann Publishers Inc., 1997

Digital Library

[18]

M. Couceiro et al. Chasing the optimum in replicated in-memory transactional platforms via protocol adaptation. In Proc. of DSN, 2013

Digital Library

[19]

A. Ganapathi et al. Predicting Multiple Metrics for Queries: Better Decisions Enabled by Machine Learning. In Proc. of ICDE, 2009

Digital Library

[20]

J. Padhye et al. Modeling TCP throughput: A simple model and its empirical validation. SIGCOMM Comput. Commun. Rev., 28(4) 303--314, Oct. 1998.

Digital Library

[21]

P. Di Sanzo et al. On the analytical modeling of concurrency control algorithms for software transactional memories: The case of commit-time-locking. Performance Evaluation 69(5), May, 2012

Digital Library

Cited By

Lattuada MBarbierato EGianniti EArdagna D(2022)Optimal Resource Allocation of Cloud-Based Spark ApplicationsIEEE Transactions on Cloud Computing10.1109/TCC.2020.298568210:2(1301-1316)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TCC.2020.2985682
Lauer SRieck SWanner MFlügge W(2022)Partial Automated Multi-Pass-Welding for Thick Sheet Metal ConnectionsAnnals of Scientific Society for Assembly, Handling and Industrial Robotics 202110.1007/978-3-030-74032-0_33(399-410)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-74032-0_33
Ataie EEvangelinou AGianniti EArdagna D(2021)A Hybrid Machine Learning Approach for Performance Modeling of Cloud-Based Big Data ApplicationsThe Computer Journal10.1093/comjnl/bxab13165:12(3123-3140)Online publication date: 20-Sep-2021
https://doi.org/10.1093/comjnl/bxab131
Show More Cited By

Index Terms

Hybrid Machine Learning/Analytical Models for Performance Prediction: A Tutorial
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
      1. Modeling methodologies

Recommendations

Enhancing Performance Prediction Robustness by Combining Analytical Modeling and Machine Learning
ICPE '15: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

Classical approaches to performance prediction rely on two, typically antithetic, techniques: Machine Learning (ML) and Analytical Modeling (AM). ML takes a black box approach, whose accuracy strongly depends on the representativeness of the dataset ...
Machine Learning: The State of the Art

The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...
Suicidal behaviour prediction models using machine learning techniques: A systematic review
Abstract Background
Early detection and prediction of suicidal behaviour are key factors in suicide control. In conjunction with recent advances in the field of artificial intelligence, there is increasing research into how machine ...
Highlights
- Classification of an individual with suicidal behaviours using statistical techniques suffers from low predictive accuracy.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '15: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

January 2015

366 pages

ISBN:9781450332484

DOI:10.1145/2668930

General Chairs:
Lizy K. John
UT Austin, USA
,
Connie U. Smith
L&S Computer Technology, Inc., USA
,
Program Chairs:
Kai Sachs
SAP SE, Germany
,
Catalina M. Lladó
University of the Balearic Islands, Spain

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMETRICS: ACM Special Interest Group on Measurement and Evaluation
SIGSOFT: ACM Special Interest Group on Software Engineering
SPEC: SPEC Research Group

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 January 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

ICPE'15

Sponsor:

ICPE'15: ACM/SPEC International Conference on Performance Engineering

January 28 - February 4, 2015

Texas, Austin, USA

Acceptance Rates

ICPE '15 Paper Acceptance Rate 23 of 74 submissions, 31%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
689
Total Downloads

Downloads (Last 12 months)160
Downloads (Last 6 weeks)13

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lattuada MBarbierato EGianniti EArdagna D(2022)Optimal Resource Allocation of Cloud-Based Spark ApplicationsIEEE Transactions on Cloud Computing10.1109/TCC.2020.298568210:2(1301-1316)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TCC.2020.2985682
Lauer SRieck SWanner MFlügge W(2022)Partial Automated Multi-Pass-Welding for Thick Sheet Metal ConnectionsAnnals of Scientific Society for Assembly, Handling and Industrial Robotics 202110.1007/978-3-030-74032-0_33(399-410)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-74032-0_33
Ataie EEvangelinou AGianniti EArdagna D(2021)A Hybrid Machine Learning Approach for Performance Modeling of Cloud-Based Big Data ApplicationsThe Computer Journal10.1093/comjnl/bxab13165:12(3123-3140)Online publication date: 20-Sep-2021
https://doi.org/10.1093/comjnl/bxab131
Barbierato EGribaudo MSerazzi G(2020)Multi-formalism Models for Performance EngineeringFuture Internet10.3390/fi1203005012:3(50)Online publication date: 13-Mar-2020
https://doi.org/10.3390/fi12030050
Scavuzzo MNitto EArdagna D(2018)Experiences and challenges in building a data intensive system for data migrationEmpirical Software Engineering10.1007/s10664-017-9503-723:1(52-86)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s10664-017-9503-7
Ataie EGianniti EArdagna DMovaghar A(2016)A Combined Analytical Modeling Machine Learning Approach for Performance Prediction of MapReduce Jobs in Cloud Environment2016 18th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC)10.1109/SYNASC.2016.072(431-439)Online publication date: Sep-2016
https://doi.org/10.1109/SYNASC.2016.072

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten