Chico Sundermann
Thomas Thüm
Ina Schaefer
ABSTRACT
Configurable systems are widely used for families of products that share multiple configuration options. These systems often induce a large configuration space. Handling the variability of such a system is difficult without being able to measure its complexity. Several methods depend on computing the number of valid configurations, such as estimating the effort of an update or effectively reducing the variability of a system. In many cases, it is possible to map a configurable system to propositional logic. Therefore, we use #SAT in order to evaluate variability of such systems. A #SAT solver computes the number of valid assignments of a propositional formula. However, this problem is even harder than SAT. The main contribution of our work is an investigation of the scalability of off-the-shelf #SAT solvers on industrial feature models. Additionally, we examine the correlation between size of a system and the runtime of a solver computing the number of valid configurations. In this paper, we empirically evaluate nine publicly available #SAT solvers on 127 industrial feature models. Our results indicate that current solvers master a majority of the evaluated systems. However, there are large models, for which none of the evaluated solvers scales. Nevertheless, there are even larger and more complex systems for which the solvers scale.
- Mathieu Acher, Philippe Collet, Philippe Lahire, and Robert B France. 2011. Slicing feature models. In Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 424--427.Google Scholar
Digital Library
- Mathieu Acher, Philippe Collet, Philippe Lahire, and Robert B France. 2013. Familiar: A domain-specific language for large scale management of feature models. Science of Computer Programming 78, 6 (2013), 657--681.Google ScholarDigital Library
- Sofia Ananieva. 2016. Explaining Defects and Identifying Dependencies in Interrelated Feature Models. Ph.D. Dissertation. Institute of Software.Google Scholar
- Sofia Ananieva, Matthias Kowal, Thomas Thüm, and Ina Schaefer. 2016. Implicit constraints in partial feature models. In Proceedings of the 7th International Workshop on Feature-Oriented Software Development. ACM, 18--27.Google ScholarDigital Library
- Ebrahim Bagheri, Tommaso Di Noia, Dragan Gasevic, and Azzurra Ragone. 2012. Formalizing interactive staged feature model configuration. Journal of Software: Evolution and Process 24, 4 (2012), 375--400.Google ScholarCross Ref
- Roberto J Bayardo Jr and Joseph Daniel Pehoushek. 2000. Counting models using connected components. In AAAI/IAAI. 157--162.Google Scholar
- David Benavides, Sergio Segura, and Antonio Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Information Systems 35, 6 (2010), 615--636.Google ScholarDigital Library
- David Benavides, Sergio Segura, Pablo Trinidad, and Antonio Ruiz-Cortés. 2005. Using Java CSP solvers in the automated analyses of feature models. In International Summer School on Generative and Transformational Techniques in Software Engineering. Springer, 399--408.Google Scholar
- David Benavides, Pablo Trinidad, and Antonio Ruiz-Cortés. 2005. Automated reasoning on feature models. In International Conference on Advanced Information Systems Engineering. Springer, 491--503.Google ScholarDigital Library
- Armin Biere. 2008. PicoSAT essentials. Journal on Satisfiability, Boolean Modeling and Computation 4 (2008), 75--97.Google ScholarCross Ref
- Armin Biere, Marijn Heule, and Hans van Maaren. 2009. Handbook of satisfiability. Vol. 185. IOS press.Google Scholar
- Jan Burchard, Tobias Schubert, and Bernd Becker. 2015. Laissez-faire caching for parallel# SAT solving. In International Conference on Theory and Applications of Satisfiability Testing. Springer, 46--61.Google ScholarCross Ref
- Supratik Chakraborty, Kuldeep S Meel, and Moshe Y Vardi. 2013. A scalable approximate model counter. In International Conference on Principles and Practice of Constraint Programming. Springer, 200--216.Google ScholarCross Ref
- Paul C Clements, John D McGregor, and Sholom G Cohen. 2005. The structured intuitive model for product line economics (SIMPLE). Technical Report. CARNEGIE-MELLON UNIV PITTSBURGH PA SOFTWARE ENGINEERING INST.Google Scholar
- Adnan Darwiche. 2004. New advances in compiling CNF to decomposable negation normal form. In Proceedings of the 16th European Conference on Artificial Intelligence. Citeseer, 318--322.Google ScholarDigital Library
- Adnan Darwiche and Pierre Marquis. 2002. A knowledge compilation map. Journal of Artificial Intelligence Research 17 (2002), 229--264.Google ScholarCross Ref
- David Fernandez-Amoros, Ruben Heradio, Jose A Cerrada, and Carlos Cerrada. 2014. A scalable approach to exact model and commonality counting for extended feature models. IEEE Transactions on Software Engineering 40, 9 (2014), 895--910.Google ScholarCross Ref
- Carla P Gomes, Ashish Sabharwal, and Bart Selman. 2006. Model counting: A new strategy for obtaining good bounds. In AAAI. 54--61.Google Scholar
- Tarik Hadzic, Sathiamoorthy Subbarayan, Rune M Jensen, Henrik R Andersen, Jesper Møller, and Henrik Hulgaard. 2004. Fast backtrack-free product configuration using a precompiled solution space representation. small 10, 1 (2004), 3.Google Scholar
- Ruben Heradio, David Fernandez-Amoros, Jose A Cerrada, and Ismael Abad. 2013. A literature review on feature diagram product counting and its usage in software product line economic models. International Journal of Software Engineering and Knowledge Engineering 23, 08 (2013), 1177--1204.Google ScholarCross Ref
- Jinbo Huang and Adnan Darwiche. 2004. Using DPLL for efficient OBDD construction. In International Conference on Theory and Applications of Satisfiability Testing. Springer, 157--172.Google ScholarDigital Library
- Jinbo Huang and Adnan Darwiche. 2005. DPLL with a trace: From SAT to knowledge compilation. In IJCAI, Vol. 5. 156--162.Google Scholar
- David S Johnson. 1992. The NP-completeness column: an ongoing guide. Journal of algorithms 13, 3 (1992), 502--524.Google ScholarDigital Library
- Christian Kastner, Thomas Thum, Gunter Saake, Janet Feigenspan, Thomas Leich, Fabian Wielgorz, and Sven Apel. 2009. FeatureIDE: A tool framework for feature-oriented software development. In Proceedings of the 31st International Conference on Software Engineering. IEEE Computer Society, 611--614.Google ScholarDigital Library
- Alexander Knüppel, Thomas Thüm, Stephan Mennicke, Jens Meinicke, and Ina Schaefer. 2017. Is there a mismatch between real-world feature models and product-line research?. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 291--302.Google ScholarDigital Library
- Sebastian Krieter, Reimar Schröter, Thomas Thüm, Wolfram Fenske, and Gunter Saake. 2016. Comparing algorithms for efficient feature-model slicing. In Proceedings of the 20th International Systems and Software Product Line Conference. ACM, 60--64.Google ScholarDigital Library
- Andreas Kübler, Christoph Zengler, and Wolfgang Küchlin. 2010. Model counting in product configuration. arXiv preprint arXiv:1007.1024 (2010).Google Scholar
- Jean-Marie Lagniez and Pierre Marquis. 2017. An Improved Decision-DNNF Compiler.. In IJCAI. 667--673.Google Scholar
- Marcilio Mendonca, Moises Branco, and Donald Cowan. 2009. SPLOT: software product lines online tools. In Proceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications. ACM, 761--762.Google Scholar
- Marcilio Mendonca, Andrzej Wąsowski, and Krzysztof Czarnecki. 2009. SAT-based analysis of feature models is easy. In Proceedings of the 13th International Software Product Line Conference. Carnegie Mellon University, 231--240.Google ScholarDigital Library
- Marcilio Mendonca, Andrzej Wasowski, Krzysztof Czarnecki, and Donald Cowan. 2008. Efficient compilation techniques for large scale feature models. In Proceedings of the 7th international conference on Generative programming and component engineering. ACM, 13--22.Google ScholarDigital Library
- Christian Muise, Sheila McIlraith, J Christopher Beck, and Eric Hsu. 2010. Fast d-DNNF Compilation with sharpSAT. In Workshops at the twenty-fourth AAAI conference on artificial intelligence.Google Scholar
- Daniel-Jesus Munoz, Jeho Oh, Mónica Pinto, Lidia Fuentes, and Don Batory. 2019. Uniform random sampling product configurations of feature models that have numerical features. In Proceedings of the 23rd International Systems and Software Product Line Conference-Volume A. ACM, 39.Google ScholarDigital Library
- Macha Nikolskaia and Antoine Rauzy. 1998. Heuristics for BDD handling of sum-of-products formulae. In Proceedings of the European Safety and Reliability Association Conference, ESREL'98. Citeseer.Google Scholar
- Jeho Oh, Don Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding near-optimal configurations in product lines by random sampling. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 61--71.Google ScholarDigital Library
- Jeho Oh, Paul Gazzillo, Don Batory, Marijn Heule, and Maggie Myers. 2019. Uniform sampling from kconfig feature models. The University of Texas at Austin, Department of Computer Science, Tech. Rep. TR-19-02 (2019).Google Scholar
- Richard Pohl, Kim Lauenroth, and Klaus Pohl. 2011. A performance comparison of contemporary algorithmic approaches for automated analysis operations on feature models. In Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 313--322.Google ScholarDigital Library
- Tian Sang, Fahiem Bacchus, Paul Beame, Henry A Kautz, and Toniann Pitassi. 2004. Combining Component Caching and Clause Learning for Effective Model Counting. SAT 4 (2004), 7th.Google Scholar
- Tian Sang, Paul Beame, and Henry Kautz. 2005. Heuristics for fast exact model counting. In International Conference on Theory and Applications of Satisfiability Testing. Springer, 226--240.Google ScholarDigital Library
- Reimar Schröter, Sebastian Krieter, Thomas Thüm, Fabian Benduhn, and Gunter Saake. 2016. Feature-model interfaces: the highway to compositional analyses of highly-configurable systems. In Proceedings of the 38th International Conference on Software Engineering. ACM, 667--678.Google ScholarDigital Library
- Thomas Thum, Christian Kastner, Sebastian Erdweg, and Norbert Siegmund. 2011. Abstract features in feature modeling. In 2011 15th International Software Product Line Conference. IEEE, 191--200.Google ScholarDigital Library
- Marc Thurley. 2006. sharpSAT-counting models with advanced component caching and implicit BCP. In International Conference on Theory and Applications of Satisfiability Testing. Springer, 424--429.Google ScholarDigital Library
- Takahisa Toda and Takehide Soh. 2016. Implementing efficient all solutions SAT solvers. Journal of Experimental Algorithmics (JEA) 21 (2016), 1--12.Google ScholarDigital Library
- Lin Xu, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2008. SATzilla: portfolio-based algorithm selection for SAT. Journal of artificial intelligence research 32 (2008), 565--606.Google ScholarCross Ref
Recommendations
Applications of #SAT Solvers on Feature Models
VaMoS '21: Proceedings of the 15th International Working Conference on Variability Modelling of Software-Intensive SystemsProduct lines are ubiquitous for managing variable systems. The variability of a product line is typically described in terms of a feature model. Analyzing a feature model gives insight into various aspects, such as the validity of a configuration of ...
SMT-based variability analyses in FeatureIDE
VaMoS '20: Proceedings of the 14th International Working Conference on Variability Modelling of Software-Intensive SystemsHandling configurable systems with thousands of configuration options is a challenging problem in research and industry. One of the most common approaches to manage the configuration options of large systems is variability modelling. The verification ...
Uniform Random Sampling Product Configurations of Feature Models That Have Numerical Features
SPLC '19: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume AAnalyses of Software Product Lines (SPLs) rely on automated solvers to navigate complex dependencies among features and find legal configurations. Often these analyses do not support numerical features with constraints because propositional formulas use ...
Comments