skip to main content
10.1145/3461001.3471146acmconferencesArticle/Chapter ViewAbstractPublication PagessplcConference Proceedingsconference-collections
research-article

Monte Carlo tree search for feature model analyses: a general framework for decision-making

Published: 06 September 2021 Publication History

Abstract

The colossal solution spaces of most configurable systems make intractable their exhaustive exploration. Accordingly, relevant analyses remain open research problems. There exist analyses alternatives such as SAT solving or constraint programming. However, none of them have explored simulation-based methods. Monte Carlo-based decision making is a simulation-based method for dealing with colossal solution spaces using randomness. This paper proposes a conceptual framework that tackles various of those analyses using Monte Carlo methods, which have proven to succeed in vast search spaces (e.g., game theory). Our general framework is described formally, and its flexibility to cope with a diversity of analysis problems is discussed (e.g., finding defective configurations, feature model reverse engineering or getting optimal performance configurations). Additionally, we present a Python implementation of the framework that shows the feasibility of our proposal. With this contribution, we envision that different problems can be addressed using Monte Carlo simulations and that our framework can be used to advance the state of the art a step forward.

References

[1]
Muhammad Sarmad Ali, Muhammad Ali Babar, and Klaus Schmid. 2009. A Comparative Survey of Economic Models for Software Product Lines. In 35th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE Computer Society, 275--278.
[2]
Sven Apel, Don S. Batory, Christian Kästner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines - Concepts and Implementation. Springer.
[3]
Wesley K. G. Assunção, Roberto E. Lopez-Herrejon, Lukas Linsbauer, Silvia R. Vergilio, and Alexander Egyed. 2017. Multi-objective reverse engineering of variability-safe feature models based on code dependencies of system variants. Empir. Softw. Eng. 22, 4 (2017), 1763--1794.
[4]
Wesley K. G. Assunção, Roberto E. Lopez-Herrejon, Lukas Linsbauer, Silvia R. Vergilio, and Alexander Egyed. 2017. Reengineering legacy applications into software product lines: a systematic mapping. Empir. Softw. Eng. 22, 6 (2017), 2972--3016.
[5]
Wesley K. G. Assunção, Silvia R. Vergilio, and Roberto E. Lopez-Herrejon. 2020. Automatic extraction of product line architecture and feature models from UML class diagram variants. Inf. Softw. Technol. 117 (2020).
[6]
Satomi Baba, Yongjoon Joe, Atsushi Iwasaki, and Makoto Yokoo. 2011. Real-Time Solving of Quantified CSPs Based on Monte-Carlo Game Tree Search. In 22nd International Joint Conference on Artificial Intelligence (IJCAI). Barcelona, Spain, 655--661.
[7]
Don S. Batory. 2005. Feature Models, Grammars, and Propositional Formulas. In 9th International Conference on Software Product Lines (SPLC). 7--20.
[8]
David Benavides. 2019. Variability Modelling and Analysis During 30 Years. In From Software Engineering to Formal Methods and Tools, and Back (LNCS, Vol. 11865). Springer, 365--373.
[9]
David Benavides, Sergio Segura, and Antonio Ruiz Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Inf. Syst. 35, 6 (2010), 615--636.
[10]
Megha Bhushan, José Ángel Galindo Duarte, Piyush Samant, Ashok Kumar, and Arun Negi. 2021. Classifying and resolving software product line redundancies using an ontological first-order logic rule based method. Expert Systems with Applications 168 (2021), 114167.
[11]
Chris Bogart, Christian Kästner, James Herbsleb, and Ferdian Thung. 2021. When and how to make breaking changes. ACM Trans. Softw. Eng. Methodol 1, 1 (2021).
[12]
C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (2012), 1--43.
[13]
Murray Cantor. 2011. Calculating and improving ROI in software and system programs. Commun. ACM 54, 9 (2011), 121--130.
[14]
Guillaume Chaslot, Sander Bakkes, Istvan Szita, and Pieter Spronck. 2008. Monte-Carlo Tree Search: A New Framework for Game AI. In 4th Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE). AAAI Press, 216--217.
[15]
Guillaume Chaslot, Mark Winands, H. Herik, Jos Uiterwijk, and Bruno Bouzy. 2008. Progressive Strategies for Monte-Carlo Tree Search. New Mathematics and Natural Computation 04 (2008), 343--357.
[16]
Guillaume Chaslot, Mark H. M. Winands, and H. Jaap van den Herik. 2008. Parallel Monte-Carlo Tree Search. In 6th International Conference on Computers and Games (CG) (LNCS, Vol. 5131). Springer, 60--71.
[17]
Muhammad Umar Chaudhry and Jee-Hyong Lee. 2018. MOTiFS: Monte Carlo Tree Search Based Feature Selection. Entropy 20, 5 (2018).
[18]
Krzysztof Czarnecki, Steven She, and Andrzej Wasowski. 2008. Sample Spaces and Feature Models: There and Back Again. In 12th International Conference on Software Product Lines (SPLC). IEEE Computer Society, 22--31.
[19]
Krzysztof Czarnecki and Andrzej Wasowski. 2007. Feature Diagrams and Logics: There and Back Again. In 11th International Conference on Software Product Lines (SPLC). IEEE Computer Society, 23--34.
[20]
Thiago do Nascimento Ferreira, Jackson A. Prado Lima, Andrei Strickler, Josiel Neumann Kuk, Silvia Regina Vergilio, and Aurora T. R. Pozo. 2017. Hyper-Heuristic Based Product Selection for Software Product Line Testing. IEEE Comput. Intell. Mag. 12, 2 (2017), 34--45.
[21]
Alexander Felfernig, Rouven Walter, José Angel Galindo, David Benavides, Seda Polat Erdeniz, Müslüm Atas, and Stefan Reiterer. 2018. Anytime diagnosis for reconfiguration. J. Intell. Inf. Syst. 51, 1 (2018), 161--182.
[22]
David Fernández-Amorós, Ruben Heradio, Carlos Cerrada, Enrique Herrera-Viedma, and Manuel J. Cobo. 2017. Towards Taming Variability Models in the Wild. In 16th International Conference on New Trends in Intelligent Software Methodologies, Tools and Techniques (SoMeT) (Frontiers in Artificial Intelligence and Applications, Vol. 297). 454-465.
[23]
David Fernández-Amorós, Ruben Heradio, José Antonio Cerrada, and Carlos Cerrada. 2014. A Scalable Approach to Exact Model and Commonality Counting for Extended Feature Models. IEEE Trans. Software Eng. 40, 9 (2014), 895--910.
[24]
José A. Galindo and David Benavides. 2020. A Python framework for the automated analysis of feature models: A first step to integrate community efforts. In 24th ACM International Systems and Software Product Line Conference (SPLC), Vol. B. ACM, Montreal, Canada, 52--55.
[25]
José Angel Galindo, David Benavides, Pablo Trinidad, Antonio Manuel Gutiérrez-Fernández, and Antonio Ruiz-Cortés. 2019. Automated analysis of feature models: Quo vadis? Computing 101, 5 (2019), 387--433.
[26]
José Angel Galindo, Hamilton A. Turner, David Benavides, and Jules White. 2016. Testing variability-intensive systems using automated analysis: an application to Android. Softw. Qual. J. 24, 2 (2016), 365--405.
[27]
Dharmalingam Ganesan, Jens Knodel, Ronny Kolb, Uwe Haury, and Gerald Meier. 2007. Comparing Costs and Benefits of Different Test Strategies for a Software Product Line: A Study from Testo AG. In 11th International Conference on Software Product Lines (SPLC). 74--83.
[28]
Paul Gazzillo, Ugur Koc, ThanhVu Nguyen, and Shiyi Wei. 2018. Localizing configurations in highly-configurable systems. In 22nd International Systems and Software Product Line Conference (SPLC), Vol. 1. ACM, Gothenburg, Sweden, 269--273.
[29]
Sylvain Gelly and David Silver. 2011. Monte-Carlo tree search and rapid action value estimation in computer Go. Artificial Intelligence 175, 11 (2011), 1856--1875.
[30]
Jianmei Guo, Jia Hui Liang, Kai Shi, Dingyu Yang, Jingsong Zhang, Krzysztof Czarnecki, Vijay Ganesh, and Huiqun Yu. 2019. SMTIBEA: a hybrid multi-objective optimization algorithm for configuring large constrained software product lines. Softw. Syst. Model. 18, 2 (2019), 1447--1466.
[31]
Axel Halin, Alexandre Nuttinck, Mathieu Acher, Xavier Devroey, Gilles Perrouin, and Benoit Baudry. 2019. Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack. Empir. Softw. Eng. 24, 2 (2019), 674--717.
[32]
Ruben Heradio, David Fernández-Amorós, José A. Galindo, and David Benavides. 2020. Uniform and scalable SAT-sampling for configurable systems. In 24th ACM International Systems and Software Product Line Conference (SPLC), Vol. A. ACM, Montreal, Canada, 17:1--17:11.
[33]
Ruben Heradio, David Fernández-Amorós, Christoph Mayr-Dorn, and Alexander Egyed. 2019. Supporting the statistical analysis of variability models. In 41st International Conference on Software Engineering (ICSE). IEEE / ACM, 843--853.
[34]
Ruben Heradio, David Fernández-Amorós, Luis Torre-Cubillo, and Alberto Pérez García-Plaza. 2012. Improving the accuracy of COPLIMO to estimate the payoff of a software product line. Expert Syst. Appl. 39, 9 (2012), 7919--7928.
[35]
Ruben Heradio, Hector Perez-Morago, David Fernández-Amorós, Roberto Bean, Francisco Javier Cabrerizo, Carlos Cerrada, and Enrique Herrera-Viedma. 2016. Binary Decision Diagram Algorithms to Perform Hard Analysis Operations on Variability Models. In 15th International Conference on New Trends in Software Methodologies, Tools and Techniques (SoMeT) (Frontiers in Artificial Intelligence and Applications, Vol.286). 139--154.
[36]
Cyrille Jégourel, Axel Legay, and Sean Sedwards. 2013. Importance Splitting for Statistical Model Checking Rare Properties. In 25th International Conference on Computer Aided Verification (CAV) (LNCS, Vol. 8044). Springer, 576--591.
[37]
Reza Karimpour and Guenther Ruhe. 2017. Evolutionary robust optimization for software product line scoping: An explorative study. Comput. Lang. Syst. Struct. 47 (2017), 189--210.
[38]
Levente Kocsis and Csaba Szepesvári. 2006. Bandit Based Monte-Carlo Planning. In Machine Learning: ECML 2006. Berlin, Heidelberg, 282--293.
[39]
Dirk P. Kroese, Tim Brereton, Thomas Taimre, and Zdravko I. Botev. 2014. Why the Monte Carlo method is so important today. WIREs Computational Statistics 6, 6 (2014), 386--392.
[40]
Lukas Linsbauer, Roberto Erick Lopez-Herrejon, and Alexander Egyed. 2017. Variability extraction and modeling for product variants. Softw. Syst. Model. 16, 4 (2017), 1179--1199.
[41]
Roberto Erick Lopez-Herrejon, Sheny Illescas, and Alexander Egyed. 2018. A systematic mapping study of information visualization for software product line engineering. J. Softw. Evol. Process. 30, 2 (2018).
[42]
Roberto E. Lopez-Herrejon, Lukas Linsbauer, and Alexander Egyed. 2015. A systematic mapping study of search-based software engineering for software product lines. Inf. Softw. Technol. 61 (2015), 33--51.
[43]
Roberto Erick Lopez-Herrejon, Lukas Linsbauer, José Angel Galindo, José Antonio Parejo, David Benavides, Sergio Segura, and Alexander Egyed. 2015. An assessment of search-based techniques for reverse engineering feature models. J. Syst. Softw. 103 (2015), 353--369.
[44]
Maira Marques, Jocelyn Simmonds, Pedro O. Rossel, and María Cecilia Bastarrica. 2019. Software product line evolution: A systematic literature review. Inf. Softw. Technol. 105 (2019), 190--208.
[45]
Marzio Marseguerra, Enrico Zio, and Luca Podofillini. 2007. Genetic Algorithms and Monte Carlo Simulation for the Optimization of System Design and Operation. In Computational Intelligence in Reliability Engineering: Evolutionary Techniques in Reliability Analysis and Optimization. Studies in Computational Intelligence, Vol. 39. Springer, 101--150.
[46]
Jabier Martinez, Tewfik Ziadi, Raúl Mazo, Tegawendé F. Bissyandé, Jacques Klein, and Yves Le Traon. 2014. Feature Relations Graphs: A Visualisation Paradigm for Feature Constraints in Software Product Lines. In 2nd IEEE Working Conference on Software Visualization (VISSOFT). IEEE Computer Society, Victoria, BC, Canada, 50--59.
[47]
R. Munos. 2014. From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning.
[48]
Daniel-Jesus Munoz, Jeho Oh, Mónica Pinto, Lidia Fuentes, and Don Batory. 2019. Uniform random sampling product configurations of feature models that have numerical features. In 23rd International Systems and Software Product Line Conference (SPLC), Vol. A. 289--301.
[49]
Vivek Nair, Tim Menzies, Norbert Siegmund, and Sven Apel. 2017. Using bad learners to find good configurations. In 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). ACM, Paderborn, Germany, 257--267.
[50]
Hootan Nakhost and Martin Müller. 2009. Monte-Carlo Exploration for Deterministic Planning. In 21st International Jont Conference on Artifical Intelligence (IJCAI). Morgan Kaufmann Publishers Inc., San Francisco, USA, 1766--1771.
[51]
Alexander Nöhrer and Alexander Egyed. 2011. Optimizing User Guidance during Decision-Making. In 15th International Conference on Software Product Lines (SPLC). IEEE Computer Society, 25--34.
[52]
Makoto Nonaka and Liming Zhu. 2007. Impact of Architecture and Quality Investment in Software Product Line Development. In 11th International Conference on Software Product Lines (SPLC). IEEE Computer Society, Kyoto, Japan, 63--73.
[53]
Jeho Oh, Don S. Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding near-optimal configurations in product lines by random sampling. In 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). Paderborn, Germany, 61--71.
[54]
Juliana Alves Pereira, Lucas Maciel, Thiago F. Noronha, and Eduardo Figueiredo. 2018. Heuristic and exact algorithms for product configuration in software product lines. In 22nd International Systems and Software Product Line Conference (SPLC), Vol. 1. Gothenburg, Sweden, 247.
[55]
Juliana Alves Pereira, Jabier Martinez, Hari Kumar Gurudu, Sebastian Krieter, and Gunter Saake. 2018. Visual guidance for product line configuration using recommendations and non-functional properties. In 33rd Annual ACM Symposium on Applied Computing (SAC). 2058--2065.
[56]
Juliana Alves Pereira, Pawel Matuszyk, Sebastian Krieter, Myra Spiliopoulou, and Gunter Saake. 2016. A feature-based personalized recommender system for product-line configuration. In ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE). ACM, Pau, France, 120--131.
[57]
Tobias Pett, Thomas Thüm, Tobias Runge, Sebastian Krieter, Malte Lochau, and Ina Schaefer. 2019. Product sampling for product lines: the scalability challenge. In 23rd International Systems and Software Product Line Conference (SPLC). ACM, Paris, France, 14:1--14:6.
[58]
Matias Pol'la, Agustina Buccella, and Alejandra Cechich. 2020. Analysis of variability models: a systematic literature review. Software and Systems Modeling (2020), 1--35.
[59]
Simon Poulding and Robert Feldt. 2015. Heuristic Model Checking Using a Monte-Carlo Tree Search Algorithm. In Annual Conference on Genetic and Evolutionary Computation (GECCO) (Madrid, Spain). Association for Computing Machinery, 1359--1366.
[60]
Alessandro Previti, Raghuram Ramanujan, Marco Schaerf, and Bart Selman. 2011. Monte-Carlo Style UCT Search for Boolean Satisfiability. In Artificial Intelligence Around Man and Beyond (AI*IA). Berlin, Heidelberg, 177--188.
[61]
Jorge Rodas-Silva, José Angel Galindo, Jorge García-Gutiérrez, and David Benavides. 2019. Selection of Software Product Line Implementation Components Using Recommender Systems: An Application to Wordpress. IEEE Access 7 (2019), 69226--69245.
[62]
Stuart J. Russell and Peter Norvig. 2020. Artificial Intelligence - A Modern Approach, Fourth edition. Pearson Education. http://vig.pearsoned.com/store/product/1,1207,store-12521_isbn-0136042597,00.html
[63]
Abdel Salam Sayyad, Joseph Ingram, Tim Menzies, and Hany H. Ammar. 2013. Optimum feature selection in software product lines: Let your model and values guide your search. In 1st International Workshop on Combining Modelling and Search-Based Software Engineering (CMSBSE@ICSE). San Francisco, CA, USA, 22--27.
[64]
Steven She, Rafael Lotufo, Thorsten Berger, Andrzej Wasowski, and Krzysztof Czarnecki. 2010. The Variability Model of The Linux Kernel. In 4th International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS) (ICB-Research Report, Vol. 37). Universität Duisburg-Essen, Linz, Austria, 45--51. http://www.vamos-workshop.net/proceedings/VaMoS_2010_Proceedings.pdf
[65]
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of Go without human knowledge. Nature 550, 7676 (2017), 354--359.
[66]
E. S. Steinmetz and M. Gini. 2020. More Trees or Larger Trees: Parallelizing Monte Carlo Tree Search. IEEE Transactions on Games (2020), 1--1.
[67]
Chico Sundermann, Thomas Thüm, and Ina Schaefer. 2020. Evaluating #SAT solvers on industrial feature models. In 14th International Working Conference on Variability Modelling of Software-Intensive Systems (VaMoS). ACM, Magdeburg, Germany, 3:1--3:9.
[68]
Y. Tanabe, K. Yoshizoe, and H. Imai. 2009. A study on security evaluation methodology for image-based biometrics authentication systems. In 3rd IEEE International Conference on Biometrics: Theory, Applications, and Systems. 1--6.
[69]
Paul Temple, José Angel Galindo, Mathieu Acher, and Jean-Marc Jézéquel. 2016. Using machine learning to infer constraints for product lines. In 20th International Systems and Software Product Line Conference (SPLC). ACM, Beijing, China, 209--218.
[70]
Thomas Thüm. 2020. A BDD for Linux?: the knowledge compilation challenge for variability. In 24th ACM International Systems and Software Product Line Conference (SPLC), Vol. A. 16:1--16:6.
[71]
Cristian Vidal-Silva, José A. Galindo, Jesús Giráldez-Cru, and David Benavides. 2021. Automated Completion of Partial Configurations as a Diagnosis Task Using FastDiag to Improve Performance. In Intelligent Systems in Industrial Applications. Springer International Publishing, Cham, 107--117.
[72]
Leland Wilkinson and Michael Friendly. 2009. The History of the Cluster Heat Map. The American Statistician 63, 2 (2009), 179--184.
[73]
Bang Wong. 2010. Color coding. Nature Methods 7, 8 (2010), 573--573.
[74]
Yinxing Xue, Jinghui Zhong, Tian Huat Tan, Yang Liu, Wentong Cai, Manman Chen, and Jun Sun. 2016. IBED: Combining IBEA and DE for optimal feature selection in software product line engineering. Appl. Soft Comput. 49 (2016), 1215--1231.
[75]
Hitesh Yadav, A. Charan Kumari, and Rita Chhikara. 2020. Feature selection optimisation of software product line using metaheuristic techniques. International Journal of Embedded Systems 13, 1 (2020), 50--64.

Cited By

View all
  • (2024)Operationalizing Decision Tables: A Step-by-Step Framework for Efficient Software Product Line CustomizationEvaluation of Novel Approaches to Software Engineering10.1007/978-3-031-64182-4_8(165-188)Online publication date: 10-Jul-2024
  • (2023)A Monte Carlo tree search conceptual framework for feature model analysesJournal of Systems and Software10.1016/j.jss.2022.111551195:COnline publication date: 1-Jan-2023
  • (2023)Mastering uncertainty in performance estimations of configurable software systemsEmpirical Software Engineering10.1007/s10664-022-10250-228:2Online publication date: 19-Jan-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SPLC '21: Proceedings of the 25th ACM International Systems and Software Product Line Conference - Volume A
September 2021
239 pages
ISBN:9781450384698
DOI:10.1145/3461001
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 September 2021

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. configurable systems
  2. feature models
  3. monte carlo tree search
  4. software product lines
  5. variability modeling

Qualifiers

  • Research-article

Funding Sources

  • EU FEDER program and MINECO
  • Spanish Government
  • Junta de Andalucia

Conference

SPLC '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 167 of 463 submissions, 36%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)19
  • Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Operationalizing Decision Tables: A Step-by-Step Framework for Efficient Software Product Line CustomizationEvaluation of Novel Approaches to Software Engineering10.1007/978-3-031-64182-4_8(165-188)Online publication date: 10-Jul-2024
  • (2023)A Monte Carlo tree search conceptual framework for feature model analysesJournal of Systems and Software10.1016/j.jss.2022.111551195:COnline publication date: 1-Jan-2023
  • (2023)Mastering uncertainty in performance estimations of configurable software systemsEmpirical Software Engineering10.1007/s10664-022-10250-228:2Online publication date: 19-Jan-2023
  • (2022)Improving Variabilty Analysis through Scenario-Based Incompatibility DetectionInformation10.3390/info1303014913:3(149)Online publication date: 11-Mar-2022
  • (2022)Applications of Monte-Carlo Tree Search in Software Engineering2022 International Conference on Mechatronics, Electronics and Automotive Engineering (ICMEAE)10.1109/ICMEAE58636.2022.00032(145-150)Online publication date: 5-Dec-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media