Skip to main content
SpringerLink
Log in

On the relation of control-flow and performance feature interactions: a case study

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Detecting feature interactions is imperative for accurately predicting performance of highly-configurable systems. State-of-the-art performance prediction techniques rely on supervised machine learning for detecting feature interactions, which, in turn, relies on time-consuming performance measurements to obtain training data. By providing information about potentially interacting features, we can reduce the number of required performance measurements and make the overall performance prediction process more time efficient. We expect that information about potentially interacting features can be obtained by analyzing the source code of a highly-configurable system, which is computationally cheaper than performing multiple performance measurements. To this end, we conducted an in-depth qualitative case study on two real-world systems (mbedTLS and SQLite), in which we explored the relation between internal (precisely control-flow) feature interactions, detected through static program analysis, and external (precisely performance) feature interactions, detected by performance-prediction techniques using performance measurements. We found that a relation exists that can potentially be exploited to predict performance interactions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://tls.mbed.org/, https://www.sqlite.org/

  2. https://openvpn.net/, https://www.mozilla.org/

  3. 10 combinations with 2 features, 10 with 3, 5 with 4, and 1 with 5.

  4. http://fosd.net/TypeChef/

  5. http://fosd.net/SPLConqueror/

  6. http://orange.biolab.si/

  7. http://sqlite.org/speed.html

References

  • Apel S, Batory D, Kästner C, Saake G (2013a) Feature-Oriented Software Product Lines. Springer, Berlin

  • Apel S, Kolesnikov S, Siegmund N, Kästner C, Garvin B (2013b) Exploring feature interactions in the wild: The new feature-interaction challenge. In: Proceedings of FOSD Workshop. ACM, pp 1–8

  • Borgelt C (2012) Frequent item set mining. Wiley Interdiscip Rev: Data Mining Knowl Discov 2(6):437–456

    Google Scholar 

  • Bruns G (2005) Foundations for features. In: Feature Interactions in Telecommunications and Software Systems VIII. IOS Press, pp 3–11

  • Duong T, Rizzo J (2011) Here come the ⊕ ninjas. https://web.archive.org/web/20150630133214/http://www.hpcc.ecs.soton.ac.uk/dan/talks/bullrun/Beast.pdf, Accessed: 2019-03-01

  • Ferreira G, Kästner C, Pfeffer J, Apel S (2015) Characterizing complexity of highly-configurable systems with variational call graphs: Analyzing configuration options interactions complexity in function calls. In: Proceedings Hotsos, ACM, pp 17:1–2

  • Flyvbjerg B (2006) Five misunderstandings about case-study research. Qual Inq 12(2):219–245

    Article  Google Scholar 

  • Garvin BJ, Cohen M (2011) Feature interaction faults revisited: An exploratory study. In: Proceedings of ISSRE. IEEE, pp 90–99

  • Guo J, Czarnecki K, Apel S, Siegmund N, Wasowski A (2013) Variability-aware performance prediction: A statistical learning approach. In: Proceedings of ASE. IEEE, pp 301–311

  • Guo J, Yang D, Siegmund N, Apel S, Sarkar A, Valov P, Czarnecki K, Wasowski A, Yu H (2018) Data-efficient performance learning for configurable systems. Empir Softw Eng 23(3):1826–1867

    Article  Google Scholar 

  • Jaccard P (1912) The distribution of the flora in the alpine zone. New phytologist 11(2):37–50

    Article  Google Scholar 

  • Kaltenecker C, Grebhahn A, Siegmund N, Guo J, Apel S (2019) Distance-based sampling of software configuration spaces. In:. Proceedings of ICSE, ACM, to appear

  • Kang K, Cohen S, Hess J, Novak W, Peterson A (1990) Feature-Oriented Domain Analysis (FODA) Feasibility Study. Tech. Rep. CMU/SEI-90-TR-21, Carnegie Mellon University

  • Kästner C, Apel S, ur Rahman SS, Rosenmüller M, Batory D, Saake G (2009) On the impact of the optional feature problem: Analysis and case studies. In: Proceedings of SPLC, pp 181–190

  • Kim C, Batory D, Khurshid S (2011) Reducing combinatorics in testing product lines. In: Proceedings of AOSD. ACM, pp 57–68

  • Kolesnikov S, Siegmund N, Kästner C, Grebhahn A, Apel S (2018) Tradeoffs in modeling performance of highly configurable software systems. Software and Systems Modeling (SoSyM) pp 1–19, online first

  • Lillack M, Kästner C, Bodden E (2018) Tracking load-time configuration options. IEEE Trans Softw Eng (TSE) 44(12):1269–1291

    Article  Google Scholar 

  • Maqbool O, Babri H (2007) Hierarchical clustering for software architecture recovery. IEEE Trans Softw Eng 33(11):759–780

    Article  Google Scholar 

  • Medeiros F, Kästner C, Ribeiro M, Gheyi R, Apel S (2016) A comparison of 10 sampling algorithms for configurable systems. In: Proceedings of ICSE. ACM, pp 643–654

  • Meinicke J, Wong C, Kästner C, Thüm T, Saake G (2016) On essential configuration complexity: measuring interactions in highly-configurable systems. In: Proceedings of ASE. ACM Press, pp 483–494

  • Nair V, Menzies T, Siegmund N, Apel S (2017) Using bad learners to find good configurations. In: Proceedings of ESEC/FSE, pp 257–267

  • Nair V, Menzies T, Siegmund N, Apel S (2018a) Faster discovery of faster system configurations with spectral learning. Autom Softw Eng 25(2):247–277

  • Nair V, Yu Z, Menzies T, Siegmund N, Apel S (2018b) Finding faster configurations using FLASH. IEEE Transactions on Software Engineering (TSE), pp 1–1, https://doi.org/10.1109/TSE.2018.2870895, online first

  • Nguyen T, Koc U, Cheng J, Foster JS, Porter A (2016) iGen: Dynamic interaction inference for configurable software. In: Proceedings of FSE. ACM, pp 655-665

  • Passos L, Queiroz R, Mukelabai M, Berger T, Apel S, Czarnecki K, Padilla J (2018) A study of feature scattering in the Linux kernel. IEEE Transactions on Software Engineering (TSE) Online first

  • Qiao Y, He J, Yang Y, Ji L (2013) Analyzing malware by abstracting the frequent itemsets in API call sequences. In: Proceedings of TrustCom. IEEE, pp 265–270

  • Reisner E, Song C, Ma K, Foster JS, Porter A (2010) Using symbolic evaluation to understand behavior in configurable software systems. In: Proceedings of ICSE. ACM, pp 445–454

  • Sarkar A, Guo J, Siegmund N, Apel S, Czarnecki K (2015) Cost-efficient sampling for performance prediction of configurable systems. In: Proceedings of ASE. IEEE, pp 342–352

  • Shull F, Singer J, Sjøberg D (2007) Guide to Advanced Empirical Software Engineering. Springer, Berlin

    Google Scholar 

  • Siegmund N, Kolesnikov S, Kästner C, Apel S, Batory D, Rosenmüller M, Saake G (2012) Predicting performance via automated feature-interaction detection. In: Proceedings of ICSE. IEEE, pp 167–177

  • Siegmund N, von Rhein A, Apel S (2013a) Family-based performance measurement. In: Proceedings of GPCE. ACM, pp 95–104

  • Siegmund N, Rosenmu̇ller M, Kȧstner C, Giarrusso P, Apel S, Kolesnikov S (2013b) Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption. Inf Softw Technol 55(3):491–507

  • Siegmund N, Grebhahn A, Apel S, Kästner C (2015) Performance-influence models for highly configurable systems. In: Proceedings of ESEC/FSE. ACM, pp 284–294

  • Soares LR, Meinicke J, Nadi S, Kästner C, de Almeida ES (2018) Exploring Feature interactions without specifications: A controlled experiment. In: Proceedings of GPCE. ACM Press, pp 41–52

  • Tartler R, Lohmann D, Dietrich C, Egger C, Sincero J (2012) Configuration coverage in the analysis of large-scale system software. SIGOPS Oper Syst Rev (ACM OSR) 45(3):10–14

    Article  Google Scholar 

  • Thereska E, Doebel B, Zheng A, Nobel P (2010) Practical performance models for complex, popular applications. SIGMETRICS Perform Eval Rev 38(1):1–12

    Article  Google Scholar 

  • von Rhein A, Grebhahn A, Apel S, Siegmund N, Beyer D, Berger T (2015) Presence-condition simplification in highly configurable systems. In: Proceedings of ICSE. IEEE, vol 1, pp 178–188

  • von Rhein A, Liebig J, Janker A, Kästner C, Apel S (2018) Variability-aware static analysis at scale: An empirical study. ACM Trans Softw Eng Methodol (TOSEM) 27(4):18:1–18:33

    Google Scholar 

  • Westermann D, Happe J, Krebs R, Farahbod R (2012) Automated inference of goal-oriented performance prediction functions. In: Proceedings of ASE. ACM, pp 190–199

  • Yin R (2003) Case Study Research–Design and Methods. Sage, Newbury Park

    Google Scholar 

  • Zave P (2009) Modularity in distributed feature composition. Software Requirements and Design: The Work of Michael Jackson, pp 267–290

  • Zhang Y, Guo J, Blais E, Czarnecki K (2015) Performance prediction of configurable software systems by Fourier learning. In: Proceedings of ASE. IEEE, pp 365–373

Download references

Acknowledgements

Kolesnikov’s, and Apel’s work has been supported by the German Research Foundation (AP 206/5, AP 206/6, AP 206/7, AP 206/11) and by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) project No. 849928. Siegmund’s work has been supported by the German Research Foundation under the contracts SI 2171/2 and SI 2171/3. Kästner’s work has been supported in part by the National Science Foundation (awards 1318808, 1552944, and 1717022), the Science of Security Lablet (H9823014C0140), and AFRL and DARPA (FA8750-16-2-0042).

Author information

Authors and Affiliations

  1. University of Passau, Passau, Germany

    Sergiy Kolesnikov

  2. Bauhaus-University Weimar, Weimar, Germany

    Norbert Siegmund

  3. Carnegie Mellon University, Pittsburgh, PA, USA

    Christian Kästner

  4. Saarland University, Saarbrücken, Germany

    Sven Apel

Authors
  1. Sergiy Kolesnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Norbert Siegmund
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Kästner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sven Apel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergiy Kolesnikov.

Additional information

Communicated by: Tao Yue

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kolesnikov, S., Siegmund, N., Kästner, C. et al. On the relation of control-flow and performance feature interactions: a case study. Empir Software Eng 24, 2410–2437 (2019). https://doi.org/10.1007/s10664-019-09705-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09705-w

Keywords

Search

Navigation