skip to main content
10.1145/3555228.3555232acmotherconferencesArticle/Chapter ViewAbstractPublication PagessbesConference Proceedingsconference-collections
research-article

Validating an Interactive Ranking Operator for NSGA-II to Support the Optimization of Software Engineering Problems

Published: 05 October 2022 Publication History

Abstract

Search-Based Software Engineering (SBSE) has been beneficial for optimizing the solution of several Software Engineering (SE) problems. The incorporation of Decision Makers (DM) preferences during the search process may help the algorithms to find more adequate solutions for their profiles. Some interactive approaches allow the DM to evaluate solutions, rating them with scores during the search process. These scores represent the adequacy level of the solutions in relation to the DM preferences and should influence the evolution of the search algorithm. In previous work, we proposed an interactive ranking operator for NSGA-II to support the complete prioritization of solutions for any SE problem domain. Although this operator worked satisfactorily in an application example, its validation is required so that it can be used in real application contexts. In this sense, we instantiated the interactive ranking operator for NSGA-II presented in previous work, and we conducted an exploratory study with a twofold goal: (i) validate the impact in the ranking of solutions, and (ii) check the diversity of them. To accomplish such goals, we made statistical tests such as correlation and regression analysis using quality metrics for Product Line Architecture (PLA) Design. The results pointed out that the interactive ranking operator can properly deal with the DM preferences, giving a greater chance of surviving to those solutions with higher scores, without compromising their diversity.

References

[1]
Haldun Akoglu. 2018. User’s guide to correlation coefficients. Turkish journal of emergency medicine 18 (2018), 91–93.
[2]
Vahid Alizadeh and Marouane Kessentini. 2018. Reducing interactive refactoring effort via clustering-based multi-objective search. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). 464–474.
[3]
Boukhdhir Amal, Marouane Kessentini, Slim Bechikh, Josselin Dea, and Lamjed Ben Said. 2014. On the use of machine learning and search-based software engineering for ill-defined fitness function: a case study on software refactoring. In International Symposium on Search Based Software Engineering. 31–45.
[4]
Allysson Allex Araújo, Matheus Paixao, Italo Yeltsin, Altino Dantas, and Jerffeson Souza. 2017. An architecture based on interactive optimization and machine learning applied to the next release problem. Automated Software Engineering 24 (2017), 623–671.
[5]
Andrea Arcuri and Lionel Briand. 2014. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24 (2014), 219–250.
[6]
Carlos Vinicius Bindewald, Willian Marques Freire, Aline Maria Malachini Miotto Amaral, and Thelma Elita Colanzi. 2020. Supporting user preferences in search-based product line architecture design using Machine Learning. In XIV Brazilian Symposium on Software Components, Architectures, and Reuse.
[7]
Jürgen Branke, Jurgen Branke, Kalyanmoy Deb, Kaisa Miettinen, and Roman Slowiński. 2008. Multiobjective optimization: Interactive and evolutionary approaches. Springer Science & Business Media.
[8]
Jürgen Branke and Kalyanmoy Deb. 2005. Integrating user preferences into evolutionary multi-objective optimization. In Knowledge incorporation in evolutionary computation. Springer, 461–477.
[9]
Thelma Elita Colanzi, Wesley K. G. Assunção, Silvia R. Vergilio, Paulo Roberto Farah, and Giovani Guizzo. 2020. The Symposium on Search-Based Software Engineering: Past, Present and Future. Information and Software Technology 127 (2020), 106372.
[10]
Thelma Elita Colanzi, Silvia Regina Vergilio, Itana M. S. Gimenes, and Willian Nalepa Oizumi. 2014. A Search-Based Approach for Software Product Line Design. In International Software Product Line Conference (SPLC). 237–241.
[11]
Altino Dantas, Italo Yeltsin, Allysson Allex Araújo, and Jerffeson Souza. 2015. Interactive software release planning with preferences base. In International Symposium on Search Based Software Engineering. 341–346.
[12]
Richard Dawkins 1996. The blind watchmaker: Why the evidence of evolution reveals a universe without design. WW Norton & Company.
[13]
Kalyanmoy Deb. 2003. Multi-objective evolutionary algorithms: Introducing bias among Pareto-optimal solutions. In Advances in evolutionary computing. Springer, 263–292.
[14]
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6 (2002), 182–197.
[15]
Thiago Nascimento Ferreira, Allysson Allex Araújo, Altino Dantas Basílio Neto, and Jerffeson Teixeira de Souza. 2016. Incorporating user preferences in ant colony optimization for the next release problem. Applied Software Computing 49 (2016), 1283–1296.
[16]
Thiago Nascimento Ferreira, Silvia Regina Vergilio, and Jerffeson T. de Souza. 2017. Incorporating user preferences in search-based software engineering: A systematic mapping study. Information and Software Technology 90 (2017), 55–69.
[17]
Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering statistics using R.
[18]
Willian Marques Freire, Mamoru Massago, Arthur Cattaneo Zavadski, Aline M. M. M. Amaral, and Thelma Elita Colanzi. 2020. OPLA-Tool v2.0: a Tool for Product Line Architecture Design Optimization. In 34th Proc. of the Brazilian Symposium on Software Engineering (SBES). 818–823.
[19]
Mark Harman and Bryan F. Jones. 2001. Search-based software engineering. Information and Soft. Technology 43 (2001), 833–839.
[20]
Martin Höst, Björn Regnell, and Claes Wohlin. 2000. Using students as subjects—a comparative study of students and professionals in lead-time impact assessment. Empirical Software Engineering 5 (2000), 201–214.
[21]
Wael Kessentini and Vahid Alizadeh. 2020. Interactive metamodel/model co-evolution using unsupervised learning and multi-objective search. In Proc. of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems. 68–78.
[22]
Wael Kessentini, Manuel Wimmer, and Houari Sahraoui. 2018. Integrating the designer in-the-loop for metamodel/model co-evolution via interactive computational search. In Proc. of the 21th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems. 101–111.
[23]
William Kruskal and Allen Wallis. 1952. Use of ranks in one-criterion variance analysis. Journal of the American statistical Association 47 (1952), 583–621.
[24]
Bogdan Marculescu, Robert Feldt, Richard Torkar, and Simon Poulding. 2018. Transferring interactive search-based software testing to industry. Journal of Systems and Software 142 (2018), 156–170.
[25]
Mohamed Wiem Mkaouer, Marouane Kessentini, Slim Bechikh, Kalyanmoy Deb, and Mel Ó Cinnéide. 2014. Recommendation system for software refactoring using innovization and interactive dynamic optimization. In Proc. of the 29th ACM/IEEE international conference on Automated software engineering. 331–336.
[26]
Joao Choma Neto, Tatiane Gaieski, Aline Miotto Amaral, and Thelma Elita Colanzi. 2018. Quanti-Qualitative Analysis of a Memetic Algorithm to Optimize Product Line Architecture Design. In 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI). 498–505.
[27]
Vilfredo Pareto and Ann S Schwier. 1927. Manual of political economy Tr. by Ann S. Schwier.
[28]
Antonio Mauricio Pitangueira. 2015. Incorporating preferences from multiple stakeholders in software requirements selection an interactive search-based approach. In 23rd International Requirements Engineering Conference. 382–387.
[29]
Aurora Ramirez, Jose Raul Romero, and Christopher Simons. 2018. A systematic review of interaction in search-based software engineering. IEEE Transactions on Software Engineering(2018).
[30]
Aurora Ramirez, José Raúl Romero, and Sebastian Ventura. 2018. Interactive multi-objective evolutionary optimization of software architectures. Information Sciences 463(2018), 92–109.
[31]
Soumaya Rebai, Vahid Alizadeh, Marouane Kessentini, Houcem Fehri, and Rick Kazman. 2020. Enabling decision and objective space exploration for interactive multi-objective refactoring. IEEE Transactions on Software Engineering(2020).
[32]
Cláudia Tupan Rosa, Willian Marques Freire, Aline M. M. M. Amaral, and Thelma Elita Colanzi. 2022. Towards an Interactive Ranking Operator for NSGA-II. In Genetic and Evolutionary Computation Conference Companion (GECCO). 794–797.
[33]
Raphael Saraiva, Allysson Allex Araújo, Altino Dantas, Italo Yeltsin, and Jerffeson Souza. 2017. Incorporating decision maker’s preferences in a multi-objective approach for the software release planning. Journal of the Brazilian Computer Society 23 (2017), 11.
[34]
SEI. 2009. Software Engineering Institute - The Arcade Game Maker Pedagogical Product Line. https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=485941. Accessed in 2018 August.
[35]
Samuel Shapiro and Martin Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52(1965), 591–611.
[36]
Chris Simons, Jeremy Singer, and David R. White. 2015. Search-Based Refactoring: Metrics Are Not Enough. In 7th International Symposium on Search-Based Software Engineering (SSBSE). 47–61.
[37]
Christopher L Simons and Ian C Parmee. 2008. User-centered, evolutionary search in conceptual software design. In 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence). 869–876.
[38]
Christopher L. Simons, Ian C. Parmee, and Rhys Gwynllyw. 2010. Interactive, evolutionary search in upstream object-oriented class design. IEEE Transactions on Software Engineering 36 (2010), 798–816.
[39]
Karl Sims. 1994. Evolving virtual creatures. In Proc. of the 21st annual conference on Computer graphics and interactive techniques. 15–22.
[40]
Hideyuki Takagi. 2001. Interactive evolutionary computation: Fusion of the capabilities of EC optimization and human evaluation. Proc. IEEE 89(2001), 1275–1296.
[41]
Otimizes (UEM). 2022. Experimental package. https://doi.org/10.6084/m9.figshare.19615977.v1
[42]
Milan Zeleny and James L. Cochrane. 1973. Multiple criteria decision making. University of South Carolina Press.
[43]
Eckart Zitzler, Dimo Brockhoff, and Lothar Thiele. 2007. The hypervolume indicator revisited: On the design of Pareto-compliant indicators via weighted integration. In International Conference on Evolutionary Multi-Criterion Optimization. 862–876.
[44]
Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M Fonseca, and Viviane Grunert Da Fonseca. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on evolutionary computation 7 (2003), 117–132.

Cited By

View all
  • (2024)A comprehensive survey on interactive evolutionary computation in the first two decades of the 21st centuryApplied Soft Computing10.1016/j.asoc.2024.111950164(111950)Online publication date: Oct-2024
  • (2023)Feature selection in an interactive search-based PLA design approachProceedings of the 17th Brazilian Symposium on Software Components, Architectures, and Reuse10.1145/3622748.3622750(11-20)Online publication date: 25-Sep-2023
  • (2023)Studying the Influence and Distribution of the Human Effort in a Hybrid Fitness Function for Search-Based Model-Driven EngineeringIEEE Transactions on Software Engineering10.1109/TSE.2023.332973049:12(5189-5202)Online publication date: 10-Nov-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SBES '22: Proceedings of the XXXVI Brazilian Symposium on Software Engineering
October 2022
457 pages
ISBN:9781450397353
DOI:10.1145/3555228
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Interactive SBSE
  2. NSGA-II ranking operator.
  3. Optimization for Software Engineering problems

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SBES 2022
SBES 2022: XXXVI Brazilian Symposium on Software Engineering
October 5 - 7, 2022
Virtual Event, Brazil

Acceptance Rates

Overall Acceptance Rate 147 of 427 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)16
  • Downloads (Last 6 weeks)1
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A comprehensive survey on interactive evolutionary computation in the first two decades of the 21st centuryApplied Soft Computing10.1016/j.asoc.2024.111950164(111950)Online publication date: Oct-2024
  • (2023)Feature selection in an interactive search-based PLA design approachProceedings of the 17th Brazilian Symposium on Software Components, Architectures, and Reuse10.1145/3622748.3622750(11-20)Online publication date: 25-Sep-2023
  • (2023)Studying the Influence and Distribution of the Human Effort in a Hybrid Fitness Function for Search-Based Model-Driven EngineeringIEEE Transactions on Software Engineering10.1109/TSE.2023.332973049:12(5189-5202)Online publication date: 10-Nov-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media