skip to main content
10.1145/3302333.3302335acmotherconferencesArticle/Chapter ViewAbstractPublication PagesvamosConference Proceedingsconference-collections
invited-talk

Challenges and Insights from Optimizing Configurable Software Systems

Published: 06 February 2019 Publication History

Abstract

Configuring a software system to optimize non-functional properties is a hard task. There are dozens to thousands of configuration options that can affect performance, energy consumption, and other attributes of the resulting program. Even worse, options may interact, such that their combined presence (or absence) has an influence on a non-functional property.
In this talk, I report on our experiences with learning different performance models based on a multitude of sampling techniques. The goal is to raise awareness of the distinct challenges in this domain: constraints among options, the exponential search space, and suitable sampling and learning techniques. I show a variety of approaches including their strengths and weaknesses and close the talk with new challenges relevant for our community: changing environments and reproducibility.

References

[1]
Sven Apel, Sergiy Kolesnikov, Norbert Siegmund, Christian Kästner, and Brady Garvin. 2013. Exploring Feature Interactions in the Wild: The New Feature-Interaction Challenge. In Proceedings of the International Workshop on Feature-Oriented Software Development (FOSD). ACM, 1--8.
[2]
Alexander Grebhahn, Sebastian Kuckuk, Christian Schmitt, Harald Köstler, Norbert Siegmund, Sven Apel, Frank Hannig, and Jürgen Teich. 2014. Experiments on Optimizing the Performance of Stencil Codes with SPL Conqueror. Parallel Processing Letters (PPL) 24, 3 (Sept. 2014).
[3]
Jianmei Guo, Krzysztof Czarnecki, Sven Apel, Norbert Siegmund, and Andrzej Wasowski. 2013. Variability-Aware Performance Prediction: A Statistical Learning Approach. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering(ASE). IEEE, 301--311.
[4]
Pooyan Jamshidi, Norbert Siegmund, Miguel Velez, Christian Kästner, Akshay Patel, and Yuvraj Agarwal. 2017. Transfer Learning for Performance Modeling of Configurable Systems: An Exploratory Analysis. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). ACM Press, 497--508.
[5]
Pooyan Jamshidi, Miguel Velez, Christian Kästner, and Norbert Siegmund. 2018. Learning to Sample: Exploiting Similarities Across Environments to Learn Performance Models for Configurable Systems. In Proceedings of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, 71--82.
[6]
Pooyan Jamshidi, Miguel Velez, Christian Kästner, Norbert Siegmund, and Prassad Kawthekar. 2017. Transfer Learning for Improving Model Predictions in Highly Configurable Software. In Proceedings of the 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS). IEEE, 31--41.
[7]
Christian Kaltenecker, Alexander Grebhahn, Norbert Siegmund, Jianmei Guo, and Sven Apel. 2019. Distance-Based Sampling of Software Configuration Spaces. In Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE). ACM.
[8]
Vivek Nair, Tim Menzies, Norbert Siegmund, and Sven Apel. 2017. Using Bad Learners to find Good Configurations. In Proceedings of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering(ESEC/FSE). ACM Press, 257--267.
[9]
Vivek Nair, Tim Menzies, Norbert Siegmund, and Sven Apel. 2018. Faster Discovery of Faster System Configurations with Spectral Learning. Automated Software Engineering Journal 25 (2018), 247--277. Issue 2.
[10]
Jeho Oh, Don Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding Near-Optimal Configurations in Product Lines by Random Sampling. In Proceedings of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM Press, 61--71.
[11]
Atri Sarkar, Jianmei Guo, Norbert Siegmund, Sven Apel, and Krzysztof Czarnecki. 2015. Cost-Efficient Sampling for Performance Prediction of Configurable Systems. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering(ASE). IEEE, 342--352.
[12]
Janet Siegmund, Norbert Siegmund, and Sven Apel. 2015. Views on Internal and External Validity in Empirical Software Engineering. In Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE). IEEE, 9--19.
[13]
Norbert Siegmund, Alexander Grebhahn, Sven Apel, and Christian Kästner. 2015. Performance-Influence Models for Highly Configurable Systems. In Proceedings of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM Press, 284--294.
[14]
Norbert Siegmund, Sergiy Kolesnikov, Christian Kästner, Sven Apel, Don Batory, Marko Rosenmüller, and Gunter Saake. 2012. Predicting Performance via Automated Feature-Interaction Detection. In Proceedings of the IEEE/ACM International Conference on Software Engineering(ICSE). IEEE, 167--177.
[15]
Norbert Siegmund, Marko Rosenmüller, Christian Kästner, Paolo Giarrusso, Sven Apel, and Sergiy Kolesnikov. 2011. Scalable Prediction of Non-functional Properties in Software Product Lines. In Proceedings of the Software Product Line Conference (SPLC). IEEE, 160--169.
[16]
Norbert Siegmund, Marko Rosenmüller, Christian Kästner, Paolo Giarrusso, Sven Apel, and Sergiy Kolesnikov. 2013. Scalable Prediction of Non-functional Properties in Software Product Lines: Footprint and Memory Consumption. Information and Software Technology (IST) 55, 3 (2013), 491--507.
[17]
Norbert Siegmund, Marko Rosenmüller, Martin Kuhlemann, Christian Kästner, Sven Apel, and Gunter Saake. 2012. SPL Conqueror: Toward Optimization of Non-functional Properties in Software Product Lines. Software Quality Journal -- Special Issue on Quality Engineering for Software Product Lines (SQJ) 20, 3--4 (2012), 487--517.
[18]
Norbert Siegmund, Stefan Sobernig, and Sven Apel. 2017. Attributed Variability Models: Outside the Comfort Zone. In Proceedings of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering(ESEC/FSE). ACM Press, 268--278.
[19]
Norbert Siegmund, Alexander von Rhein, and Sven Apel. 2013. Family-Based Performance Measurement. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences (GPCE). ACM, 95--104.
[20]
Alexander von Rhein, Alexander Grebhahn, Sven Apel, Norbert Siegmund, Dirk Beyer, and Thorsten Berger. 2015. Presence-Condition Simplification in Highly Configurable Systems. In Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE). IEEE, 178--188.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
VaMoS '19: Proceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems
February 2019
116 pages
ISBN:9781450366489
DOI:10.1145/3302333
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

In-Cooperation

  • FWO: Fund for Scientific Research - Flanders (Belgium)
  • FNRS: Fonds National de la Recherche Scientifique

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 February 2019

Check for updates

Author Tags

  1. Configurable Software Systems
  2. Learning Prediction
  3. Performance
  4. Sampling

Qualifiers

  • Invited-talk
  • Research
  • Refereed limited

Funding Sources

Conference

VAMOS '19

Acceptance Rates

VaMoS '19 Paper Acceptance Rate 14 of 24 submissions, 58%;
Overall Acceptance Rate 66 of 147 submissions, 45%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 91
    Total Downloads
  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media