skip to main content
10.1145/2110147.2110155acmotherconferencesArticle/Chapter ViewAbstractPublication PagesvamosConference Proceedingsconference-collections
research-article

BeTTy: benchmarking and testing on the automated analysis of feature models

Published: 25 January 2012 Publication History

Abstract

The automated analysis of feature models is a flourishing research topic that has called the attention of both researchers and practitioners during the last two decades. During this time, the number of tools and techniques enabling the analysis of feature models has increased and also their complexity. In this scenario, the lack of specific testing mechanisms to assess the correctness and good performance of analysis tools is becoming a major obstacle hindering the development of tools and affecting their quality and reliability. In this paper, we present BeTTy, a framework for BEnchmarking and TesTing on the analYsis of feature models. Among other features, BeTTy enables the automated detection of faults in feature model analysis tools. Also, it supports the generation of motivating test data to evaluate the performance of analysis tools in both average and pessimistic cases. Part of the functionality of the framework is provided through a web-based interface facilitating the random generation of both classic and attributed feature models.

References

[1]
D. Benavides, S. Segura, and A. Ruiz-Cortés. Automated analysis of feature models 20 years later: A literature review. Information Systems, 35(6):615--636, 2010.
[2]
D. Benavides, S. Segura, P. Trinidad, and A. Ruiz-Cortés. A first step towards a framework for the automated analysis of feature models. In Managing Variability for Software Product Lines: Working With Variability Mechanisms, 2006.
[3]
FaMa Tool Suite. http://www.isa.us.es/fama/, accessed November 2011.
[4]
J. A. Galindo, D. Benavides, and S. Segura. Debian packages repositories as software product line models. towards automated analysis. In Proceeding of the First International Workshop on Automated Configuration and Tailoring of Applications (ACOTA), 2010.
[5]
Graphviz. http://www.graphviz.org/, accessed November 2011.
[6]
G. Kapfhammer. The Computer Science Handbook, chapter Software Testing. CRC Press, 2nd edition, June, 2004.
[7]
Apache Math. Apache math. http://commons.apache.org/math/, accessed November 2011.
[8]
M. Mendonca, D. D. Cowan, W. Malyk, and T. Oliveira. Collaborative product configuration: Formalization and efficient algorithms for dependency analysis. Journal of Software, 3(2):69--82, 2008.
[9]
S. Segura. Functional and Performance Testing of Feature Model Analysis Tools. Extending the FaMa Ecosystem. PhD thesis, Dept. of Computer Languages and Systems, University of Seville, 2011.
[10]
S. Segura, D. Benavides, and A. Ruiz-Cortés. Functional testing of feature model analysis tools: a test suite. Software, IET, 5(1):70--82, february 2011.
[11]
S. Segura, Robert M. Hierons, D. Benavides, and A. Ruiz-Cortés. Automated metamorphic testing on the analyses of feature models. Information and Software Technology, 53(3):245--258, 2011.
[12]
S. Segura, Robert M. Hierons, D. Benavides, and A. Ruiz-Cortés. Mutation testing on an object-oriented framework: An experience report. Information and Software Technology Special Issue on Mutation Testing, 2011.
[13]
S. Segura, JA. Parejo, Robert M. Hierons, D. Benavides, and A. Ruiz-Cortés. ETHOM: An evolutionary algorithm for optimized feature models generation. Tech Report ISA-2011-TR-03 (v. 1.0), Applied Software Engineering Research Group, 2011.
[14]
S. Segura and A. Ruiz-Cortés. Benchmarking on the automated analyses of feature models: A preliminary roadmap. In Third International Workshop on Variability Modelling of Software-intensive Systems, pages 137--143, Seville, Spain, 2009.
[15]
S. She, R. Lotufo, T. Berger, A. Wasowski, and K. Czarnecki. The variability model of the linux kernel. In Fourth International Workshop on Variability Modelling of Software-intensive Systems (VAMOS'10), Linz, Austria, January 2010.
[16]
S. She, R. Lotufo, T. Berger, A. Wasowski, and K. Czarnecki. Reverse engineering feature models. In Proceedings of the 27th International Conference on Software Engineering, pages 461--470, 2011.
[17]
S. P. L. O. T.: Software Product Lines Online Tools. http://www.splot-research.org/, accessed November 2011.
[18]
T. Thüm, D. Batory, and C. Kästner. Reasoning about edits to feature models. In International Conference on Software Engineering, pages 254--264, 2009.
[19]
P. Trinidad, A. Ruiz-Cortés, D. Benavides, and S. Segura. Three-dimensional feature diagrams visualization. In 2nd International Workshop on Visualisation in Software Product Line Engineering (ViSPLE 2008), 2008.
[20]
E. J. Weyuker. On testing non-testable programs. The Computer Journal, 25(4):465--470, 1982.
[21]
J. White, B. Dougherty, and D. Schmidt. Selecting highly optimal architectural feature sets with filtered cartesian flattening. Journal of Systems and Software, 82(8):1268--1284, 2009.

Cited By

View all
  • (2024)Generating Feature Models with UVL's Full ExpressivenessProceedings of the 28th ACM International Systems and Software Product Line Conference10.1145/3646548.3676602(61-65)Online publication date: 2-Sep-2024
  • (2023)Automated Test Suite Generation for Software Product Lines Based on Quality-Diversity OptimizationACM Transactions on Software Engineering and Methodology10.1145/362815833:2(1-52)Online publication date: 22-Dec-2023
  • (2023)FLAMAProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume B10.1145/3579028.3609008(16-19)Online publication date: 28-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
VaMoS '12: Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive Systems
January 2012
193 pages
ISBN:9781450310581
DOI:10.1145/2110147
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 January 2012

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automated analysis
  2. benchmarking
  3. feature models
  4. testing
  5. validation

Qualifiers

  • Research-article

Funding Sources

  • Andalusian Government
  • Spanish Government under CICYT project SETI

Conference

VaMoS '12

Acceptance Rates

Overall Acceptance Rate 66 of 147 submissions, 45%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)22
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Generating Feature Models with UVL's Full ExpressivenessProceedings of the 28th ACM International Systems and Software Product Line Conference10.1145/3646548.3676602(61-65)Online publication date: 2-Sep-2024
  • (2023)Automated Test Suite Generation for Software Product Lines Based on Quality-Diversity OptimizationACM Transactions on Software Engineering and Methodology10.1145/362815833:2(1-52)Online publication date: 22-Dec-2023
  • (2023)FLAMAProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume B10.1145/3579028.3609008(16-19)Online publication date: 28-Aug-2023
  • (2023)Large Language Models to generate meaningful feature model instancesProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608973(15-26)Online publication date: 28-Aug-2023
  • (2023)A benchmark generator framework for evolving variant-rich softwareJournal of Systems and Software10.1016/j.jss.2023.111736203(111736)Online publication date: Sep-2023
  • (2023)A Monte Carlo tree search conceptual framework for feature model analysesJournal of Systems and Software10.1016/j.jss.2022.111551195:COnline publication date: 1-Jan-2023
  • (2022)Reasoning Algorithms on Feature Modeling—A Systematic Mapping StudyApplied Sciences10.3390/app1211556312:11(5563)Online publication date: 30-May-2022
  • (2022)Test case aggregation for efficient feature model testingProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume B10.1145/3503229.3547046(174-177)Online publication date: 12-Sep-2022
  • (2022)Safe Dynamic Reconfiguration of Concurrent Component-based Applications2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C54293.2022.00027(108-111)Online publication date: Mar-2022
  • (2022)Exploring Functionality and Efficiency of Feature Model Product Configuration SolutionsIEEE Access10.1109/ACCESS.2022.323144910(134318-134332)Online publication date: 2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media