Skip to main content
SpringerLink
Log in

SMTIBEA: a hybrid multi-objective optimization algorithm for configuring large constrained software product lines

  • Regular Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

A key challenge to software product line engineering is to explore a huge space of various products and to find optimal or near-optimal solutions that satisfy all predefined constraints and balance multiple often competing objectives. To address this challenge, we propose a hybrid multi-objective optimization algorithm called SMTIBEA that combines the indicator-based evolutionary algorithm (IBEA) with the satisfiability modulo theories (SMT) solving. We evaluated the proposed algorithm on five large, constrained, real-world SPLs. Compared to the state-of-the-art, our approach significantly extends the expressiveness of constraints and simultaneously achieves a comparable performance. Furthermore, we investigate the performance influence of the SMT solving on two evolutionary operators of the IBEA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Apel, S., Batory, D.S., Kästner, C., Saake, G.: Feature-Oriented Software Product Lines—Concepts and Implementation. Springer, Berlin (2013)

    Book  Google Scholar 

  2. Arcuri, A., Briand, L.C.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: Proceedings of 33rd International Conference on Software Engineering (ICSE), pp. 1–10 (2011)

  3. Batory, D.: Feature models, grammars, and propositional formulas. In: Proceedings of 9th International Software Product Line Conference (SPLC), pp. 7–20 (2005)

  4. Benavides, D., Segura, S., Cortés, A.: Automated analysis of feature models 20 years later: a literature review. Inf. Syst. 35(6), 615–636 (2010)

    Article  Google Scholar 

  5. Berger, T., She, S., Lotufo, R., Wasowski, A., Czarnecki, K.: A study of variability models and languages in the systems software domain. IEEE Trans. Softw. Eng. 39(12), 1611–1640 (2013)

    Article  Google Scholar 

  6. Brockhoff, D., Friedrich, T., Neumann, F.: Analyzing hypervolume indicator based algorithms. In: Proceedings of 10th International Conference on Parallel Problem Solving from Nature (PPSN), pp. 651–660 (2008)

  7. Clements, P., Northrop, L.: Software Product Lines: Practices and Patterns. Addison-Wesley, Boston (2001)

    Google Scholar 

  8. Czarnecki, K., Eisenecker, U.: Generative Programming: Methods, Tools, and Applications. Addison-Wesley, Boston (2000)

    Google Scholar 

  9. de Moura, L., Bjørner, N.: Satisfiability modulo theories: introduction and applications. Commun. ACM 54(9), 69–77 (2011)

    Article  Google Scholar 

  10. de Moura, L., Bjørner, N.: Z3: An Efficient SMT Solver. In: Proceedings of 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 337–340 (2008)

  11. Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  12. Deb, K., Mohan, M., Mishra, S.: Towards a quick computation of well-spread Pareto-optimal solutions. In: Proceedings of Second International Conference on Evolutionary Multi-Criterion Optimization (EMO), pp. 222–236 (2003)

  13. Durillo, J.J., Nebro, A.J.: jMetal: a java framework for multi-objective optimization. Adv. Eng. Softw. 42, 760–771 (2011)

    Article  Google Scholar 

  14. Durillo, J.J., Nebro, A.J., Alba, E.: The jMetal framework for multi-objective optimization: design and architecture. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC), pp. 4138–4325. Barcelona, Spain (2010)

  15. Eén, N., Sörensson, N.: Translating pseudo-boolean constraints into SAT. J. Satisf. Boolean Model. Comput. 2(1–4), 1–26 (2006)

    MATH  Google Scholar 

  16. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Natural Computing Series. Springer, Berlin (2003)

    Book  MATH  Google Scholar 

  17. Gavanelli, M.: An algorithm for multi-criteria optimization in CSPs. In: Proceedings of 15th European Conference on Artificial Intelligence (ECAI), pp. 136–140 (2002)

  18. Guo, J., Czarnecki, K., Apel, S., Siegmund, N., Wąsowski, A.: Variability-aware performance prediction: a statistical learning approach. In: Proceedings of 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 301–311 (2013)

  19. Guo, J., White, J., Wang, G., Li, J., Wang, Y.: A genetic algorithm for optimized feature selection with resource constraints in software product lines. J. Syst. Softw. 84(12), 2208–2221 (2011)

    Article  Google Scholar 

  20. Guo, J., Zulkoski, E., Olaechea, R., Rayside, D., Czarnecki, K., Apel, S., Atlee, J.M.: Scaling exact multi-objective combinatorial optimization by parallelization. In: Proceedings of 29th ACM/IEEE International Conference on Automated Software Engineering (ASE), pp. 409–420 (2014)

  21. Harman, M.: The current state and future of search based software engineering. In: Proceedings of International Workshop on the Future of Software Engineering (FOSE), pp. 342–357 (2007)

  22. Harman, M., Jia, Y., Krinke, J., Langdon, W.B., Petke, J., Zhang, Y.: Search based software engineering for software product line engineering: a survey and directions for future work. In: Proceedings of 18th International Software Product Line Conference (SPLC), pp. 5–18 (2014)

  23. Harman, M., Jones, B.F.: Search-based software engineering. Inf. Softw. Technol. 43(14), 833–839 (2001)

    Article  Google Scholar 

  24. Harman, M., Mansouri, S., Zhang, Y.: Search based software engineering: a comprehensive analysis and review of trends techniques and applications. Tech. rep., King’s College London TR-09-03 (2009)

  25. Hartert, R., Schaus., P.: A support-based algorithm for the bi-objective Pareto constraint. In: Proceedings of 28th AAAI Conference on Artificial Intelligence (AAAI), pp. 2674–2679 (2014)

  26. Henard, C., Papadakis, M., Harman, M., Traon, Y.L.: Combining multi-objective search and constraint solving for configuring large software product lines. In: Proceedings of 37th IEEE/ACM International Conference on Software Engineering (ICSE), pp. 517–528 (2015)

  27. Henard, C., Papadakis, M., Perrouin, G., Klein, J., Traon, Y.L.: Multi-objective test generation for software product lines. In: Proceedings of 17th International Software Product Line Conference (SPLC), pp. 62–71 (2013)

  28. Hierons, R.M., Li, M., Liu, X., Segura, S., Zheng, W.: SIP: optimal product selection from feature models using many-objective evolutionary optimization. ACM Trans. Softw. Eng. Methodol. 25(2), 17 (2016)

    Article  Google Scholar 

  29. Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Modified distance calculation in generational distance and inverted generational distance. In: Proceedings of 8th International Conference on Evolutionary Multi-Criterion Optimization (EMO), pp. 110–125 (2015)

  30. Jackson, D.: Software Abstractions: Logic, Language, and Analysis. MIT, Cambridge (2006)

    Google Scholar 

  31. Kang, K., Cohen, S., Hess, J., Novak, W., Peterson, A.: Feature-oriented domain analysis (FODA) feasibility study. Tech. rep., CMU SEI, SEI-90-TR-21 (1990)

  32. Knowles, J., Corne, D.: On metrics for comparing nondominated sets. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC), pp. 711–716 (2002)

  33. Lu, H., Yue, T., Ali, S., Zhang, L.: Nonconformity resolving recommendations for product line configuration. In: Proceedings of Ninth International Conference on Software Testing, Verification and Validation (ICST), pp. 57–68 (2016)

  34. LVAT: Linux variability analysis tools. http://code.google.com/p/linux-variability-analysis-tools

  35. Marek, V., Truszczynski, M.: Stable models and an alternative logic programming paradigm. In: The Logic Programming Paradigm: A 25-year Perspective. Springer (1999)

  36. Mkaouer, W., Kessentini, M., Shaout, A., Koligheu, P., Bechikh, S., Deb, K., Ouni, A.: Many-objective software remodularization using NSGA-III. ACM Trans. Softw. Eng. Methodol. 24(3), 1–45 (2015)

    Article  Google Scholar 

  37. Nadi, S., Berger, T., Kästner, C., Czarnecki, K.: Where do configuration constraints stem from? An extraction approach and an empirical study. IEEE Trans. Softw. Eng. 41(8), 820–841 (2015)

    Article  Google Scholar 

  38. Olaechea, R., Rayside, D., Guo, J., Czarnecki, K.: Comparison of exact and approximate multi-objective optimization for software product lines. In: Proceedings of 18th International Software Product Line Conference (SPLC), pp. 92–101 (2014)

  39. Olaechea, R., Stewart, S., Czarnecki, K., Rayside, D.: Modeling and multi-objective optimization of quality attributes in variability-rich software. In: Proceedings of Fourth International Workshop on Nonfunctional System Properties in Domain Specific Modeling Languages (NFPinDSML), pp. 2:1–2:6 (2012)

  40. Passos, L.T., Berger, T., Novakovic, M., Czarnecki, K., Xiong, Y., Wasowski, A.: A study of non-boolean constraints in variability models of an embedded operating system. In: Proceedings of Third Workshop on Feature-Oriented Software Development (FOSD), pp. 9–16 (2011)

  41. Passos, L.T., Guo, J., Teixeira, L., Czarnecki, K., Wasowski, A., Borba, P.: Coevolution of variability models and related artifacts: a case study from the Linux kernel. In: Proceedings of 17th International Software Product Line Conference (SPLC), pp. 91–100 (2013)

  42. Passos, L.T., Teixeira, L., Dintzner, N., Apel, S., Wasowski, A., Czarnecki, K., Borba, P., Guo, J.: Coevolution of variability models and related software artifacts—a fresh look at evolution patterns in the Linux kernel. Empir. Softw. Eng. 21(4), 1744–1793 (2016)

    Article  Google Scholar 

  43. Pohl, K., Bockle, G., van der Linden, F.: Software Product line Engineering: Foundations, Principles, and Techniques. Springer, Berlin (2005)

    Book  MATH  Google Scholar 

  44. Rayside, D., Estler, H.C., Jackson, D.: A guided improvement algorithm for exact, general purpose, many-objective combinatorial optimization. Tech. rep., MIT-CSAIL-TR-2009-033 (2009)

  45. Saadatpanah, P., Famelis, M., Gorzny, J., Robinson, N., Chechik, M., Salay, R.: Comparing the effectiveness of reasoning formalisms for partial models. In: Proceedings of the Workshop on Model-Driven Engineering, Verification and Validation (MoDeVVa), pp. 41–46 (2012)

  46. Sarkar, A., Guo, J., Siegmund, N., Apel, S., Czarnecki, K.: Cost-efficient sampling for performance prediction of configurable systems. In: Proceedings of 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 342–352 (2015)

  47. Sarro, F., Petrozziello, A., Harman, M.: Multi-objective software effort estimation. In: Proceedings of 38th International Conference on Software Engineering (ICSE), pp. 619–630 (2016)

  48. Sayyad, A., Ingram, J., Menzies, T., Ammar, H.: Scalable product line configuration: A straw to break the camel’s back. In: Proceedings of 28th International Conference on Automated Software Engineering (ASE), pp. 465–474 (2013)

  49. Sayyad, A., Menzies, T., Ammar, H.: On the value of user preferences in search-based software engineering: A case study in software product lines. In: Proceedings of 35th International Conference on Software Engineering (ICSE), pp. 492–501. IEEE (2013)

  50. Sayyad, A.S., Ingram, J., Menzies, T., Ammar, H.: Optimum feature selection in software product lines: let your model and values guide your search. In: Proceedings of 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE), pp. 22–27 (2013)

  51. Siegmund, N., Kolesnikov, S., Kästner, C., Apel, S., Batory, D., Rosenmüller, M., Saake, G.: Predicting performance via automated feature-interaction detection. In: Proceedings of 34th International Conference on Software Engineering (ICSE), pp. 167–177 (2012)

  52. Tsang, E.: Foundations of constraint satisfaction. Academic, (1993)

  53. Vargha, A., Delaney, H.D.: A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25(2), 101–132 (2000)

    Google Scholar 

  54. White, J., Doughtery, B., Schmidt, D.C.: Filtered cartesian flattening: an approximation technique for optimally selecting features while adhering to resource constraints. In: Proceedings of First International Workshop on Analyses of Software Product Lines (ASPL), pp. 209–216 (2008)

  55. Wu, Z., Tang, J., Kwong, C.K., Chan, C.Y.: An optimization model for reuse scenario selection considering reliability and cost in software product line development. Int. J. Inf. Technol. Decis. Mak. 10(5), 811–841 (2011)

    Article  MATH  Google Scholar 

  56. Yen, G., He, Z.: Performance metric ensemble for multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 18(1), 131–144 (2014)

    Article  Google Scholar 

  57. Zhang, Y., Guo, J., Blais, E., Czarnecki, K.: Performance prediction of configurable software systems by Fourier learning. In: Proceedings of 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 365–373 (2015)

  58. Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Proceedings of 8th International Conference on Parallel Problem Solving from Nature (PPSN), pp. 832–842 (2004)

  59. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Proceedings of the Conference on Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems (EUROGEN), pp. 95–100 (2001)

  60. Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132 (2003)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank anonymous reviewers for their helpful comments. This research was partially supported by Shanghai Municipal Natural Science Foundation (No. 17ZR1406900), Shanghai Pujiang Talent Program (No. 17PJ1401900), Specialized Fund of Shanghai Municipal Commission of Economy and Informatization (No. 201602008), Specialized Research Fund for Doctoral Program of Higher Education (No. 20130074110015), National Natural Science Foundation of China (No. 61173048, 61602460), China Postdoctoral Science Foundation (No. 2016M600338), Natural Sciences and Engineering Research Council of Canada, and Pratt & Whitney Canada.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

    Jianmei Guo, Kai Shi & Huiqun Yu

  2. Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, Canada

    Jia Hui Liang, Krzysztof Czarnecki & Vijay Ganesh

  3. School of Electronic Information, Shanghai Dianji University, Shanghai, China

    Dingyu Yang

  4. Institute of Biochemistry and Cell Biology, Chinese Academy of Sciences, Shanghai, China

    Jingsong Zhang

Authors
  1. Jianmei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Hui Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dingyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jingsong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Krzysztof Czarnecki
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vijay Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Huiqun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jianmei Guo or Jingsong Zhang.

Additional information

Communicated by Dr Philippe Collet.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, J., Liang, J.H., Shi, K. et al. SMTIBEA: a hybrid multi-objective optimization algorithm for configuring large constrained software product lines. Softw Syst Model 18, 1447–1466 (2019). https://doi.org/10.1007/s10270-017-0610-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-017-0610-0

Keywords

Search

Navigation