Skip to main content
SpringerLink
Log in

Performance variability in software product lines: proposing theories from a case study

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

In the software product line research, product variants typically differ by their functionality and quality attributes are not purposefully varied. The goal is to study purposeful performance variability in software product lines, in particular, the motivation to vary performance, and the strategy for realizing performance variability in the product line architecture. The research method was a theory-building case study that was augmented with a systematic literature review. The case was a mobile network base station product line with capacity variability. The data collection, analysis and theorizing were conducted in several stages: the initial case study results were augmented with accounts from the literature. We constructed three theoretical models to explain and characterize performance variability in software product lines: the models aim to be generalizable beyond the single case. The results describe capacity variability in a base station product line. Thereafter, theoretical models of performance variability in software product lines in general are proposed. Performance variability is motivated by customer needs and characteristics, by trade-offs and by varying operating environment constraints. Performance variability can be realized by hardware or software means; moreover, the software can either realize performance differences in an emergent way through impacts from other variability or by utilizing purposeful varying design tactics. The results point out two differences compared with the prevailing literature. Firstly, when the customer needs and characteristics enable price differentiation, performance may be varied even with no trade-offs or production cost differences involved. Secondly, due to the dominance of feature modeling, the literature focuses on the impact management realization. However, performance variability can be realized through purposeful design tactics to downgrade the available software resources and by having more efficient hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ahnassay A, Bagheri E, Gasevic D (2013) Empirical evaluation in software product line engineering. Tech Rep TR-LS3-130084R4T, Laboratory for Systems Software and Semantics Ryerson University

  • Bagheri E, Di Noia T, Ragone A, Gasevic D (2010) Configuring software product line feature models based on stakeholders’ soft and hard requirements. In: Software Product Line Conference

  • Bagheri E, Noia TD, Gasevic D, Ragone A (2012) Formalizing interactive staged feature model configuration. J Softw Evolut Proc 24(4):375–400. doi:10.1002/smr.534

    Article  Google Scholar 

  • Barbacci M, Longstaff T, Klein M, Weinstock C (1995) Quality attributes. Tech Rep CMU/SEI-95-TR-021, SEI

  • Bartholdt J, Medak M, Oberhauser R (2009) Integrating quality modeling with feature modeling in software product lines. In: International Conference on Software Engineering Advances (ICSEA). doi:10.1109/ICSEA.2009.59

  • Bass L, Clements P, Kazman R (2003) Software Architecture in Practice, 2nd edn. Addison-Wesley

  • Belobaba P, Odoni A, Barnhart C (2009) The Global Airline Industry. Wiley

  • Benavides D, Martín-Arroyo PT, Cortés AR (2005) Automated reasoning on feature models. In: International Conference on Advanced Information Systems Engineering (CAiSE). doi:10.1007/11431855_34

  • Berntsson Svensson R, Gorschek T, Regnell B, Torkar R, Shahrokni A, Feldt R (2012) Quality requirements in industrial practice – an extended interview study at eleven companies. IEEE Trans Softw Eng 38(4):923–935. doi:10.1109/TSE.2011.47

    Article  Google Scholar 

  • Boehm B, Brown J, Kasper H, Lipow M, Macleod G, Merrit M (1978) Characteristics of Software Quality. North-Holland Publishing Company

  • Bosch J (2000) Design and Use of Software Architectures: Adapting and Evolving a Product-Line Approach. Addison-Wesley

  • Botterweck G, Thiel S, Nestor D, bin Abid S, Cawley C (2008) Visual tool support for configuring and understanding software product lines. In: Software Product Line Conference. doi:10.1109/SPLC.2008.32

  • Bu T, Chan MC, Ramjee R (2006) Connectivity, performance, and resiliency of IP-based CDMA radio access networks. IEEE Trans Mob Comput 5(8). doi:10.1109/TMC.2006.108

  • Cavalcanti RdO, de Almeida ES, Meira SR (2011) Extending the RiPLE-DE process with quality attribute variability realization. In: Joint Conference on Quality of Software Architectures and Architecting Critical Systems (QoSA-ISARCS)

  • Clements P, Northrop L (2001) Software Product Lines—Practices and Patterns. Addison-Wesley

  • Czarnecki K, Helsen S, Eisenecker UW (2005) Formalizing cardinality-based feature models and their specialization. Softw Proc Improv Pract 10(1):7–29. doi:10.1002/spip.213

    Article  Google Scholar 

  • Dubé L, Paré G (2003) Rigor in information systems positivist case research: Current practices, trends, and recommendations. MIS Q 27(4):597–635

    Google Scholar 

  • Etxeberria L, Sagardui G (2008) Variability driven quality evaluation in software product lines. In: Software Product Line Conference. doi:10.1109/SPLC.2008.37

  • Etxeberria L, Sagardui G, Belategi L (2007) Modelling variation in quality attributes. In: VaMOS

  • Fettke P, Houy C, Loos P (2010) On the relevance of design knowledge for design-oriented business and information systems engineering – conceptual foundations, application example, and implications. Bus Inf Syst Eng 2(6):347–358. doi:10.1007/s12599-010-0126-4

    Article  Google Scholar 

  • Galster M, Avgeriou P (2011) Handling variability in software architecture: Problems and implications. In: Working IEEE/IFIP Conference on Software Architecture, (WICSA). doi:10.1109/WICSA.2011.30

  • Galster M, Avgeriou P (2012) A variability viewpoint for enterprise software systems. In: Working IEEE/IFIP Conference on Software Architecture (WICSA) and European Conference on Software Architecture (ECSA). doi:10.1109/WICSA-ECSA.212.43

  • Galster M, Weyns D, Tofan D, Michalik B, Avgeriou P (2014) Variability in software systems—a systematic literature review. IEEE Trans Softw Eng 40(3):282–306. doi:10.1109/TSE.2013.56

    Article  Google Scholar 

  • Gimenes IMdS, Fantinato M, de Toledo MBF (2008) A product line for business process management. In: Software Product Line Conference. doi:10.1109/SPLC.2008.10

  • González-Baixauli B, Laguna MA, do Prado Leite JCS (2007) Using goal-models to analyze variability. In: VaMoS

  • Gregor S (2006) The nature of theory in information systems. MIS Q 30(3):611–642

    Google Scholar 

  • Guo J, White J, Wang G, Li J, Wang Y (2011) A genetic algorithm for optimized feature selection with resource constraints in software product lines. J Syst Softw 84(12). doi:10.1016/j.jss.2011.06.026

  • van Gurp J, Bosch J, Svahnberg M (2001) On the notion of variability in software product lines. In: Working IEEE/IFIP Conference on Software Architecture, (WICSA). doi:10.1109/WICSA.2001.948406

  • Hallsteinsen S, Fægri TE, Syrstad M (2003) Patterns in product family architecture design. In: Software Product Family Engineering (PFE). doi:10.1007/978-3-540-24667-1_19

  • Hallsteinsen S, Schouten G, Boot G, Fægri T (2006a) Dealing with architectural variation in productpopulations. In: Käkölä T, Dueñas JC (eds) Software Product Lines – Research Issues in Engineering and Management. Springer

  • Hallsteinsen S, Stav E, Solberg A, Floch J (2006b) Using product line techniques to build adaptive systems In: Software Product Line Conference. doi:10.1109/SPLINE.2006.1691586

  • Hevner AR, March ST, Park J, Ram S (2004) Design science in IS research. MIS Q 28(1):75–105

    Google Scholar 

  • Holma H, Toskala A (eds) (2000) WCDMA for UMTS: radio access for third generation mobile communications. Wiley

  • IEEE Std 1061-1998 (1998) IEEE standard for a software quality metrics methodology

  • IEEE Std 61012-1990 (1990) IEEE standard glossary of software engineering terminology

  • Ishida Y (2007) Software product lines approach in enterprise system development In: Software Product Line Conference

  • ISO/IEC 25010 (2011) Software engineering—product quality—part 1: Quality model

  • ISO/IEC 9126-1 (2001) Systems and software engineering—systems and software quality requirements and evaluation (SQuaRE)— system and software quality models

  • Jaring M, Bosch J (2002) Representing variability in software product lines: A case study. In: Software Product Line Conference

  • Jaring M, Krikhaar RL, Bosch J (2004) Representing variability in a family of MRI scanners. Softw: Practice and Experience 34(1):69–100

    Google Scholar 

  • Jarzabek S, Yang B, Yoeun S (2006) Addressing quality attributes in domain analysis for product lines. IEE Proc-Softw 153(2)

  • Kang K, Cohen S, Hess J, Novak W, Peterson A (1990) Feature-oriented domain analysis (FODA) feasibility study. Tech Rep CMU/SEI-90-TR-21, ADA 235785, Software Engineering Institute

  • Kang K, Lee J, Donohoe P (2002) Feature-oriented product line engineering. IEEE Softw 19(4)

  • Karatas AS, Oguztuzun H, Dogru A (2010) Mapping extended feature models to constraint logic programming over finite domains. In: Software Product Line Conference

  • Kishi T, Noda N (2000) Aspect-oriented analysis of product line architecture. In: Software Product Line Conference

  • Kishi T, Noda N, Katayama T (2001) Architectural design for evolution by analyzing requirements on quality attributes. In: Asia-Pacific Software Engineering Conference. doi:10.1109/APSEC.2001.991466

  • Kishi T, Noda N, Katayama T (2002) A method for product line scoping based on a decision-making framework. In: Software Product Line Conference

  • Kitchenham B, Pearl Brereton O, Budgen D, Turner M, Bailey J, Linkman S (2009) Systematic literature reviews in software engineering—a systematic literature review. Inf Softw Technology 51(1):7–15. doi:10.1016/j.infsof.2008.09.009

    Article  Google Scholar 

  • Kozuka N, Ishida Y (2011) Building a product line architecture for variant-rich enterprise applications using a data-oriented approach. In: Software Product Line Conference

  • Kuusela J, Savolainen J (2000) Requirements engineering for product families. In: International Conference on Software Engineering (ICSE)

  • Lee AS, Baskerville RL (2003) Generalizing generalizability in information systems research. Inf Systems Research 14(3):221–243. doi:10.1287/isre.14.3.221.16560

    Article  Google Scholar 

  • Lee K, Kang KC (2010) Using context as key driver for feature selection. In: Software Product Line Conference

  • Linden F, Bosch J, Kamsties E, Känsälä K, Krzanik L, Obbink H (2003) Software product family evaluation. In: Software Product-Family Engineering (PFE)

  • Matinlassi M (2005) Quality-driven software architecture model transformation. In: Working IEEE/IFIP Conference on Software Architecture

  • McCall JA, Richards P, Walters G (1977) Factors in software quality. Tech Rep TR-77-369, RADC

  • Mellado D, Fernández-Medina E, Piattini M (2008) Towards security requirements management for software product lines: A security domain requirements engineering process. Comput Stand Interfaces 30(6):361–371

    Article  Google Scholar 

  • Myllärniemi V, Männistö T, Raatikainen M (2006a) Quality attribute variability within a software product family architecture. In: Quality of Software Architectures (QoSA), vol 2

  • Myllärniemi V, Raatikainen M, Männistö T (2006b) Inter-organisational approach in rapid software product family development—a case study. In: International Conference on Software Reuse

  • Myllärniemi V, Raatikainen M, Männistö T (2012) A systematically conducted literature review: quality attribute variability in software product lines. In: Software Product Line Conference

  • Myllärniemi V, Savolainen J, Männistö T (2013) Performance variability in software product lines: A case study in the telecommunication domain. In: Software Product Line Conference

  • Mylopoulos J, Chung L, Nixon B (1992) Representing and using nonfunctional requirements: A process-oriented approach. IEEE Trans Softw Eng 18(6)

  • Mylopoulos J, Chung L, Liao S, Wang H, Yu E (2001) Exploring alternatives during requirements analysis. IEEE Softw 18(1):92–96

    Article  Google Scholar 

  • Niemelä E, Immonen A (2007) Capturing quality requirements of product family architecture. Inf and Softw Technology 49(11-12)

  • Niemelä E, Matinlassi M, Taulavuori A (2004) Practical evaluation of software product family architectures. In: Software Product Line Conference

  • Ognjanovic I, Mohabbati B, Gaevic D, Bagheri E, Bokovic M (2012) A metaheuristic approach for the configuration of business process families. In: International Conference on Services Computing (SCC)

  • Olaechea R, Stewart S, Czarnecki K, Rayside D (2012) Modelling and multi-objective optimization of quality attributes in variability-rich software. In: Fourth International Workshop on Nonfunctional System Properties in Domain Specific Modeling Languages

  • Patton MQ (1990) Qualitative Evaluation and Research Methods, 2nd edn. Sage Publications

  • Phillips R (2005) Pricing and Revenue Optimization. Stanford University Press

  • Regnell B, Berntsson-Svensson R, Olsson T (2008) Supporting roadmapping of quality requirements. IEEE Softw 25(2):42–47

    Article  Google Scholar 

  • Roos-Frantz F, Benavides D, Ruiz-Corts A, Heuer A, Lauenroth K (2012) Quality-aware analysis in product line engineering with the orthogonal variability model. Softw Quality J 20(3-4):519–565. doi:10.1007/s11219-011-9156-5

    Article  Google Scholar 

  • Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empirical Softw Eng 14(2):131–164. doi:10.1007/s10664-008-9102-8

    Article  Google Scholar 

  • Shadish WR, Cook TD, Campbell DT (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Miffl in Boston

  • Shaw M (2002) What makes good research in software engineering? Int J STTT 4(1):1–7. doi:10.1007/s10009-002-0083-4

    Article  Google Scholar 

  • Siegmund N, Kolesnikov S, Kastner C, Apel S, Batory D, Rosenmuller M, Saake G (2012a) Predicting performance via automated feature-interaction detection. In: International Conference on Software Engineering

  • Siegmund N, Rosenmüller M, Kuhlemann M, Kastner C, Apel S, Saake G (2012b) SPL Conqueror: Toward optimization of non-functional properties in software. Softw Quality J 20(3-4)

  • Siegmund N, Rosenmuller M, Kastner C, Giarrusso PG, Apel S, Kolesnikov SS (2013) Scalable prediction of non-functional properties in software product lines: Footprint and memory consumption. Inf Softw Technol 55(3):491–507

    Article  Google Scholar 

  • Sincero J, Schroder-Preikschat W, Spinczyk O (2009) Towards tool support for the configuration of non-functional properties in SPLs. In: Hawaii International Conference on System Sciences (HICSS). doi:10.1109/HICSS.2009.472

  • Sincero J, Schroder-Preikschat W, Spinczyk O (2010) Approaching non-functional properties of software product lines: Learning from products. In: Software Engineering Conference (APSEC). doi:10.1109/APSEC.2010.26

  • Sinnema M, Deelstra S, Nijhuis J, Bosch J (2006) Modeling dependencies in product families with COVAMOF. In: Engineering of Computer Based Systems (ECBS)

  • Smith CU, Williams LG (2002) Performance Solutions A Practical Guide to Creating Responsive, Scalable Software. Addison-Wesley

  • Soltani S, Asadi M, Gasevic D, Hatala M, Bagheri E (2012) Automated planning for feature model configuration based on functional and non-functional requirements. In: Software Product Line Conference

  • Stol KJ, Fitzgerald B (2013) Uncovering theories in software engineering SEMAT Workshop on General Theory of Software Engineering (GTSE)

  • Strauss A, Corbin J (1998) Basics of Qualitative Research, 2nd edn. Sage

  • Svahnberg M, van Gurp J, Bosch J (2005) A taxononomy of variability realization techniques. Softw—Practice and Experience 35(8)

  • Thiel S, Hein A (2002) Modelling and using product line variability in automotive systems. IEEE Softw 19(4):66–72

    Article  Google Scholar 

  • Thum T, Apel S, Kastner C, Schaefer I, Saake G (2014) A classification and survey of analysis strategies for software product lines. ACM Comput Surv 47(1). To appear

  • Tun TT, Boucher Q, Classen A, Hubaux A, Heymans P (2009) Relating requirements and feature configurations: A systematic approach. In: Software Product Line Conference

  • Urquhart C, Lehmann H, Myers MD (2010) Putting the theory back into grounded theory: guidelines for grounded theory studies in information systems. Inf Syst J 20(4):357–381. doi:10.1111/j.1365-2575.2009.00328.x

    Article  Google Scholar 

  • White J, Schmidt DC, Wuchner E, Nechypurenko A (2007) Automating product-line variant selection for mobile devices. In: Software Product Line Conference

  • White J, Dougherty B, Schmidt DC (2009) Selecting highly optimal architectural feature sets with filtered cartesian flattening. J Syst Softw 82(8)

  • Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Conference on Evaluation and Assessment in Software Engineering

  • Wohlin C, Prikladniki R (2013) Systematic literature reviews in software engineering. Inf Softw Technol 55(6):919–920. doi:10.1016/j.infsof.2013.02.002

    Article  Google Scholar 

  • Yin RK (1994) Case Study Research, 2nd edn. Sage, Thousand Oaks

    Google Scholar 

  • Yu Y, do Prado Leite JCS, Lapouchnian A, Mylopoulos J (2008) Configuring features with stakeholder goals. In: SAC

Download references

Acknowledgments

Aki Nyyssönen, Jukka Peltola, Ari Evisalmi, and Juha Timonen from Nokia are acknowledged for evaluating the validity of and commenting on our results. Anssi Karhinen and Juha Kuusela are acknowledged for ideas and comments.

Author information

Authors and Affiliations

  1. Aalto University, Finland P.O. Box 15400, FI-00076, Aalto, Espoo, Finland

    Varvana Myllärniemi & Mikko Raatikainen

  2. Danfoss Power Electronics A/S, Nordborg, Denmark

    Juha Savolainen

  3. University of Helsinki, Helsinki, Finland

    Tomi Männistö

Authors
  1. Varvana Myllärniemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juha Savolainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mikko Raatikainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tomi Männistö
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varvana Myllärniemi.

Additional information

Communicated by: Ebrahim Bagheri, David Benavides, Per Runeson and Klaus Schmid

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Myllärniemi, V., Savolainen, J., Raatikainen, M. et al. Performance variability in software product lines: proposing theories from a case study. Empir Software Eng 21, 1623–1669 (2016). https://doi.org/10.1007/s10664-014-9359-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-014-9359-z

Keywords

Search

Navigation