Abstract
In recent years, the evolutionary algorithms used in the solution of NP-Hard problems have become increasingly important. In addition, platforms and application development languages have diversified and started to be differentiated according to their intended use. However, the selection of an appropriate model development environment has become an important decision problem. This study guides the selection of suitable tools for optimization problems, especially in management science. The main objective is to identify the key attributes of the frameworks from the researcher’s point of view in management science and assign a total utility score to measure the relative importance of frameworks for evolutionary algorithms. For that reason, we propose a conjoint analysis model upon the preferences of management scientist for the appropriate framework that meets the needs in optimization problems. We also aim at providing effective usage of relevant frameworks for appropriate types of problems, facilitating the work of researchers and therefore increasing the quality of the optimization procedure. By doing so, losing time and effort resulting from the wrong platform and framework selection, as well as ineffective model results, will be avoided. Moreover, the frameworks are also evaluated by calculating the weights of criteria with one of the recent multi-criteria decision-making method called Euclidean best–worst method and compared with the findings obtained from conjoint analysis. This study not only provides review of existing software tools developed for optimization problems but also contributes to research and practice in the field of optimization algorithms in general and helps the researchers in management science for meeting their needs while searching for the appropriate framework.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The data given in Appendix 1 has been gathered by visiting the related references and website links. In case of inability to access the right information, we sent e-mails to the framework developers directly.
References
1000minds. https://www.1000minds.com/conjoint-analysis. Accessed 8 Nov 2019
Aboutorab H, Saberi M, Asadabadi MR, Hussain O, Chang E (2018) ZBWM: The Z-number extension of best worst method and its application for supplier development. Expert Syst Appl 107:115–125. https://doi.org/10.1016/j.eswa.2018.04.015
Adewumi A, Misra S, Omoregbe N, Sanz LF (2019) FOSSES: framework for open-source software evaluation and selection. Softw Pract Exper 49(5):780–812. https://doi.org/10.1002/spe.2682
Ahmadi-Javid A (2011) Anarchic society optimization: a human-inspired method. In: 2011 IEEE congress of evolutionary computation (CEC). pp 2586–2592. https://doi.org/https://doi.org/10.1109/CEC.2011.5949940
Alba E, Almeida F, Blesa M, Cabeza J, Cotta C, Díaz M et al (2002) MALLBA: a library of skeletons for combinatorial optimisation. In: European conference on parallel processing. Springer, Berlin, Heidelberg, pp 927–932
Alba E, Luque G, Garcia-Nieto J, Ordonez G, Leguizamon G (2007) MALLBA: a software library to design efficient optimisation algorithms. Int J Innov Comput Appl 1(1):74–85. https://doi.org/10.1504/IJICA.2007.013403
Alhubaishy A, Aljuhani A (2020) The best–worst method for resource allocation and task scheduling in cloud computing. In: 2020 3rd international conference on computer applications and information security (ICCAIS). pp 1–6 IEEE. https://doi.org/https://doi.org/10.1109/ICCAIS48893.2020.9096877
Alihodzic A, Tuba M (2013) Framework for bat algorithm optimization metaheuristic. In: 4th international conference on bioscience and bioinformatics (ICBB '13) Chania. Crete Island, Greece, pp 157–162
Amiri M, Tabatabaei MH, Ghahremanloo M, Keshavarz-Ghorabaee M, Zavadskas EK, Antucheviciene J (2020) A new fuzzy approach based on BWM and fuzzy preference programming for hospital performance evaluation: a case study. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2020.106279
Anjum M, Kapur PK, Agarwal V, Khatri SK (2020) Assessment of software vulnerabilities using best–worst method and two-way analysis. Int J Math Eng Manag Sci 5(2):328–342. https://doi.org/10.33889/IJMEMS.2020.5.2.02
Ashrafi SM, Dariane AB (2011) A novel and effective algorithm for numerical optimization: melody search (MS). In: 2011 11th international conference on hybrid intelligent systems (HIS). pp 109–114. https://doi.org/https://doi.org/10.1109/HIS.2011.6122089
Atmosukarto I (2011) GPLAB: software review. Genet Program Evol Mach 12(4):457–459. https://doi.org/10.1007/s10710-011-9142-5
Bai C, Kusi-Sarpong S, Badri Ahmadi H, Sarkis J (2019) Social sustainable supplier evaluation and selection: a group decision-support approach. Int J Prod Res 57(22):7046–7067. https://doi.org/10.1080/00207543.2019.1574042
Barhak J, Garrett A (2014) Population Generation from Statistics Using Genetic Algorithms with MIST+ INSPYRED. MODSIM World, 15-17. http://www.modsimworld.org/papers/2014/MODSIM2014_MIST_INSPYRED_Paper_Submit_2014_03_10.pdf. Accessed 5 Nov 2019
Benlian A (2010) Which type of software model is first choice? An AHP-based comparison of traditional, open-source, and on-demand office suites on the fulfillment of evaluation criteria. In: ECIS 2010 proceedings. p 149
Benlian A, Hess T (2011) Comparing the relative importance of evaluation criteria in proprietary and open-source enterprise application software selection—a conjoint study of ERP and office systems. Inform Syst J 21(6):503–525. https://doi.org/10.1111/j.1365-2575.2010.00357.x
Benitez-Hidalgo A, Nebro AJ, Garcia-Nieto J, Oregi I, Del Ser J (2019) jMetalPy: a python framework for multi-objective optimization with metaheuristics. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2019.100598
Block J, Fisch C, Vismara S, Andres R (2019) Private equity investment criteria: an experimental conjoint analysis of venture capital, business angels, and family offices. J Corp Finance 58:329–352. https://doi.org/10.1016/j.jcorpfin.2019.05.009
Bosch MJ, Kimber D, Leiva R (2020) The use of conjoint analysis in revealing preferences for hiring: the gender bias effect. In: Chinchilla Albiol N, Grau Grau M (eds) las Heras Maestro M. The new ideal worker contributions to management science, Springer, Cham, pp 139–160
Brans JP, Mareschal B, Vincke P (1984) PROMETHEE: a new family of outranking methods in multicriteria analysis. Oper Res 3:477–490
Brauers WKM, Zavadskas EK (2006) The MOORA method and its application to privatization in a transition economy. Control Cybern 35(2):443–468
Braun A, Schmeiser H, Schreiber F (2016) On consumer preferences and the willingness to pay for term life insurance. Eur J Oper Res 253(3):761–776. https://doi.org/10.1016/j.ejor.2016.02.023
Brownlee AE, Swan J, Özcan E, Parkes, AJ (2014) Hyperion2: a toolkit for {meta-, hyper-}heuristic research. In: Proceedings of the companion publication of the 2014 annual conference on genetic and evolutionary computation. ACM, pp 1133–1140
Brunelli M, Rezaei J (2019) A multiplicative best–worst method for multi-criteria decision making. Oper Res Lett 47(1):12–15. https://doi.org/10.1016/j.orl.2018.11.008
Cahon S, Melab N, Talbi EG (2004) Paradiseo: a framework for the reusable design of parallel and distributed metaheuristics. J Heuristics 10(3):357–380. https://doi.org/10.1023/B:HEUR.0000026900.92269.ec
Caruso EM, Rahnev DA, Banaji MR (2009) Using conjoint analysis to detect discrimination: revealing covert preferences from overt choices. Soc Cognit 27(1):128–137. https://doi.org/10.1521/soco.2009.27.1.128
Chen DN, Hu PJH, Kuo YR, Liang TP (2010a) A web-based personalized recommendation system for mobile phone selection: design, implementation, and evaluation. Expert Syst Appl 37(12):8201–8210. https://doi.org/10.1016/j.eswa.2010.05.066
Chen YH, Hsu IC, Lin CC (2010b) Website attributes that increase consumer purchase intention: a conjoint analysis. J Bus Res 63(9–10):1007–1014. https://doi.org/10.1016/j.jbusres.2009.01.023
Cricelli L, Famulari FM, Greco M, Grimaldi M (2019) Searching for the one: customer relationship management software selection. J Multi Criteria Decis Anal 27:173–188. https://doi.org/10.1002/mcda.1687
De Beukelaer H, Davenport GF, De Meyer G, Fack V (2015) JAMES: a modern object-oriented Java framework for discrete optimization using local search metaheuristics. In: 4th international symposium and 26th national conference on operational research. pp 134–138. https://doi.org/https://doi.org/10.1002/spe.2459
De Beukelaer H, Davenport GF, De Meyer G, Fack V (2017) JAMES: an object-oriented Java framework for discrete optimization using local search metaheuristics. Softw Pract Exper 47(6):921–938. https://doi.org/10.1002/spe.2459
De Rainville FM, Fortin FA, Gardner MA, Parizeau M, Gagné C (2012) Deap: a python framework for evolutionary algorithms. In: Proceedings of the 14th annual conference companion on genetic and evolutionary computation. ACM, pp 85–92
De Rainville FM, Fortin FA, Gardner MA, Parizeau M, Gagné C (2014) DEAP enabling nimbler evolutions. SIGEVOlution 6(2):17–26. https://doi.org/10.1145/2597453.2597455
Di Gaspero L, Schaerf A (2001) EASYLOCAL++: an object-oriented framework for the design of local search algorithms and metaheuristics. In: MIC’2001 4th metaheuristics international conference. pp 287–292
Di Gaspero L, Schaerf A (2003a) EasyLocal++: an object-oriented framework for the flexible design of local-search algorithms. Softw Pract Exper 33(8):733–765. https://doi.org/10.1002/spe.524
Di Gaspero L, Schaerf A (2003b) Writing local search algorithms using EASYLOCAL++ optimization software class libraries, vol 18. Springer, Boston, pp 155–175
Dong P, Zhang T, Ju Y, Wang A (2020) A novel multi-attribute decision-making framework based on Z-RIM: an illustrative example of cloud service selection. Soft Comput. https://doi.org/10.1007/s00500-020-05087-0
Dorigo M, Maniezzo V, Colorni A (1996) The ant system: optimization by a colony of cooperating agents IEEE transaction on systems. Man Cybern Part B 26(1):1–13. https://doi.org/10.1109/3477.484436
Durillo JJ, Nebro AJ (2011) jMetal: a java framework for multi-objective optimization. Adv Eng Softw 42:760–771. https://doi.org/10.1016/j.advengsoft.2011.05.014
Durillo JJ, Nebro AJ, Luna F, Dorronsoro B, Alba E (2006) jMetal: a java framework for developing multi-objective optimization metaheuristics. Departamento de Lenguajes y Ciencias de la Computación, University of Málaga, ETSI Informática, Campus de Teatinos, Tech. Rep. ITI-2006-10
Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science 1995 Japan, Nagoya. pp 39–43. https://doi.org/10.1109/mhs.1995.494215
Eldrandaly K, Naguib S (2013) A knowledge-based system for GIS software selection. Int Arab J Inf Technol 10(2):152–159
Elyasaf A, Sipper M (2014) Software review: the HeuristicLab framework. Genet Program Evol Mach 15(2):215–218. https://doi.org/10.1007/s10710-014-9214-4
Faris H, Aljarah I, Mirjalili S, Castillo PA, Guervós JJM (2016) EvoloPy: an open-source nature-inspired optimization framework in python. In: Proceedings of the 8th international joint conference on computational intelligence. pp 171–177
Fei L, Lu J, Feng Y (2020) An extended best–worst multi-criteria decision-making method by belief functions and its applications in hospital service evaluation. Comput Ind Eng 142:106355. https://doi.org/10.1016/j.cie.2020.106355
Fink A, Voß S (2003) HotFrame: a heuristic optimization framework. In: Voß S, Woodruff DL (eds) Optimization software class libraries. Operations Research Computer Science. Springer, Boston, pp 81–154
Fleischmann M (2015) Contrast and assimilation in consumer software selection decisions—an experimental study. In: Twenty-first Americas conference on information systems, Puerto Rico, 2015
Fortin FA, De Rainville FM, Gardner MA, Parizeau M, Gagné C (2012) DEAP: evolutionary algorithms made easy. J Mach Learn Res 13:2171–2175
Fumagalli L, Polenghi A, Negri E, Roda I (2019) Framework for simulation software selection. J Simul. https://doi.org/10.1080/17477778.2019.1598782
Gagné C, Parizeau M (2002) Open BEAGLE: a new versatile C++ framework for evolutionary computation. In: Proceedings of the genetic and evolutionary computation conference late breaking papers. pp 161–168
Garrett A (2012) Inspyred: bio-inspired algorithms in Python. https://pypi.python.org/pypi/inspyred. Accessed 1 Oct 2019
Gaur D, Aggarwal S (2019) Selection of software development model using TOPSIS methodology. In: Jain L, Balas VE, Johri P (eds) Data and communication networks. Advances in Intelligent Systems and Computing, vol 847. Springer, Singapore, pp 123–133
Glover F (1990) Tabu search: A tutorial. interfaces 20(4):74–94. https://doi.org/10.1287/inte.20.4.74
Green PE, Rao VR (1971) Conjoint measurement-for quantifying judgmental data. J Market Res 8(3):355–363. https://doi.org/10.1177/002224377100800312
Guo S, Zhao H (2017) Fuzzy best–worst multi-criteria decision-making method and its applications. Knowl Based Syst 121:23–31. https://doi.org/10.1016/j.knosys.2017.01.010
Gupta H (2018) Evaluating service quality of airline industry using hybrid best worst method and VIKOR. J Air Transp Manag 68:35–47. https://doi.org/10.1016/j.jairtraman.2017.06.001
Gupta H, Barua MK (2017) Supplier selection among SMEs on the basis of their green innovation ability using BWM and fuzzy TOPSIS. J Clean Prod 152:242–258. https://doi.org/10.1016/j.jclepro.2017.03.125
Hafezalkotob A, Hafezalkotob A (2017) A novel approach for combination of individual and group decisions based on fuzzy best–worst method. Appl Soft Comput 59:316–325. https://doi.org/10.1016/j.asoc.2017.05.036
Hainmueller J, Hopkins DJ (2015) The hidden American immigration consensus: a conjoint analysis of attitudes toward immigrants. Am J Polit Sci 59(3):529–548. https://doi.org/10.1111/ajps.12138
Hair JF, Black WC, Babin BJ, Anderson RE (2013) Multivariate data analysis. Pearson Higher Education, New York
Harder R (2001) OpenTS: an open source java tabu search framework. In: Proceedings of the INFORMS annual meeting, Miami
Head M, Ziolkowski N (2012) Understanding student attitudes of mobile phone features: Rethinking adoption through conjoint, cluster and SEM analyses. Comput Hum Behav 28(6):2331–2339. https://doi.org/10.1016/j.chb.2012.07.003
Hendiani S, Mahmoudi A, Liao H (2020) A multi-stage multi-criteria hierarchical decision-making approach for sustainable supplier selection. Appl Soft Comput 94:106456. https://doi.org/10.1016/j.asoc.2020.106456
Ho W, Ma X (2018) The state-of-the-art integrations and applications of the analytic hierarchy process. Eur J Oper Res 267(2):399–414. https://doi.org/10.1016/j.ejor.2017.09.007
Holland JH (1962) Outline for a logical theory of adaptive systems. J ACM 9(3):297–314. https://doi.org/10.1145/321127.321128
Huber J (2005) Conjoint analysis: how we got here and where we are (an update). In: Sawtooth software conference
Hwang CL, Yoon K (1981) Multiple attribute decision making: methods and applications: a state-of-the-art survey. Springer, New York
Ishizaka A, Labib A (2011) Review of the main developments in the analytic hierarchy process. Expert Syst Appl 38(11):14336–14345. https://doi.org/10.1016/j.eswa.2011.04.143
Jung Y, Kim S, Choi B (2016) Consumer valuation of the wearables: the case of smartwatches. Comput Hum Behav 63:899–905. https://doi.org/10.1016/j.chb.2016.06.040
KannanBalamurugan AKSA, Sasikala S (2019) A novel software package selection method using teaching-learning based optimization and multiple criteria decision making. IEEE Trans Eng Manag. https://doi.org/10.1109/TEM.2019.2918050
Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Global Optim 39(3):459–471. https://doi.org/10.1007/s10898-007-9149-x
Kashan AH (2009) League championship algorithm: a new algorithm for numerical function optimization. In: 2009 International conference of soft computing and pattern recognition. pp 43–48. https://doi.org/10.1109/SoCPaR.2009.21
Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mech 213(3–4):267–289. https://doi.org/10.1007/s00707-009-0270-4
Kazancoglu Y, Burmaoglu S (2013) ERP software selection with MCDM: application of TODIM method. Int J Bus Inf Syst 13(4):435–452. https://doi.org/10.1504/IJBIS.2013.055300
Keijzer M, Merelo JJ, Romero G, Schoenauer M (2002) Evolving Objects: a general purpose evolutionary computation library. InCollet P, Fonlupt C, Hao JK, Lutton E, Schoenauer M (eds) Artificial evolution. Lecture notes in computer science. Springer, Berlin, Heidelberg, pp 231–242
Keil M, Tiwana A (2006) Relative importance of evaluation criteria for enterprise systems: a conjoint study. Inform Syst J 16(3):237–262. https://doi.org/10.1111/j.1365-2575.2006.00218.x
Keshavarz GM, Zavadskas EK, Olfat L, Turskis Z (2015) Multi-criteria inventory classification using a new method of evaluation based on distance from average solution (EDAS). Informatica 26(3):435–451
Keshavarz GM, Zavadskas EK, Turskis Z, Antucheviciene J (2016) A new combinative distance-based assessment (codas) method for multi-criteria decision-making. Econ Comput Econ Cybern Stud Res 50(3):25–44
Kirkpatrick CD, Gelatt M, Vecchi S (1983) Optimization by simulated annealing. Science 220(4598):671–680. https://doi.org/10.1126/science.220.4598.671
Kocak H, Caglar A, Oztas GZ (2018) Euclidean best–worst method and its application. Int J Inf Tech Decis 17(05):1587–1605. https://doi.org/10.1142/S0219622018500323
Koczkodaj WW (1993) A new definition of consistency of pairwise comparisons. Math Comput Model 18(7):79–84. https://doi.org/10.1016/0895-7177(93)90059-8
Kronfeld M, Planatscher H, Zell A (2010) The EvA2 optimization framework. In: Blum C, Battiti R (eds) Learning and intelligent optimization. LION 2010. Lecture notes in computer science. Springer, Berlin, Heidelberg, pp 247–250
Kuhfeld WF (2010) Conjoint analysis. SAS Technical paper, MR H, 2010, pp 681–801. https://support.sas.com/techsup/technote/mr2010h.pdf. Accessed 8 Nov 2019
Kuzmanovic M, Savic G, Gusavac BA, Makajic-Nikolic D, Panic B (2013) A Conjoint-based approach to student evaluations of teaching performance. Expert Syst Appl 40(10):4083–4089. https://doi.org/10.1016/j.eswa.2013.01.039
Lai VS, Wong BK, Cheung W (2002) Group decision making in a multiple criteria environment: a case using the AHP in software selection. Eur J Oper Res 137(1):134–144. https://doi.org/10.1016/S0377-2217(01)00084-4
Lang JC, Widjaja T (2013) OREX-J: towards a universal software framework for the experimental analysis of optimization algorithms. OR Spectr 35(3):735–769. https://doi.org/10.1007/s00291-012-0290-7
Lin HY, Hsu PY, Sheen GJ (2007) A fuzzy-based decision-making procedure for data warehouse system selection. Expert Syst Appl 32(3):939–953. https://doi.org/10.1016/j.eswa.2006.01.031
Lo HW, Liou JJ, Wang HS, Tsai YS (2018) An integrated model for solving problems in green supplier selection and order allocation. J Clean Prod 190:339–352. https://doi.org/10.1016/j.jclepro.2018.04.105
Luce RD, Tukey JW (1964) Simultaneous conjoint measurement: a new type of fundamental measurement. J Math Psychol 1(1):1–27. https://doi.org/10.1016/0022-2496(64)90015-X
Lukasiewycz M, Glaß M, Reimann F, Teich J (2011) Opt4J: a modular framework for meta-heuristic optimization. In: Proceedings of the 13th annual conference on genetic and evolutionary computation. ACM, pp 1723–1730
Luke S (2017) ECJ then and now. In: Proceedings of the genetic and evolutionary computation conference, 2017. Berlin, pp 1223–1230
Luo SZ, Xing LN (2019) A hybrid decision making framework for personnel selection using BWM, MABAC and PROMETHEE. Int J Fuzzy Syst 21(8):2421–2434. https://doi.org/10.1007/s40815-019-00745-4
Maldonado S, Montoya R, Weber R (2015) Advanced conjoint analysis using feature selection via support vector machines. Eur J Oper Res 241(2):564–574. https://doi.org/10.1016/j.ejor.2014.09.051
Malhotra N (2010) Marketing research: an applied approach (6th Global edition). Pearson education, New York
Mann ZÁ, Szép T (2010) BCAT: a framework for analyzing the complexity of algorithms. In: IEEE 8th international symposium on intelligent systems and informatics, pp 297–302
Meißner M, Scholz SW, Decker R (2008) AHP versus ACA—an empirical comparison. In: Data analysis, machine learning and applications. Springer, Berlin, Heidelberg, pp 447–454
METSlib, https://projects.coin-or.org/metslib. Accessed 2 Sep 2019
Mi X, Tang M, Liao H, Shen W, Lev B (2019) The state-of-the-art survey on integrations and applications of the best worst method in decision making: Why, what, what for and what’s next? Omega 87:205–225. https://doi.org/10.1016/j.omega.2019.01.009
Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
Mohammadi M, Rezaei J (2020) Bayesian best–worst method: a probabilistic group decision making model. Omega 96:102075. https://doi.org/10.1016/j.omega.2019.06.001
Mou Q, Xu Z, Liao H (2016) An intuitionistic fuzzy multiplicative best–worst method for multi-criteria group decision making. Inf Sci 374:224–239. https://doi.org/10.1016/j.ins.2016.08.074
Muzyka D, Birley S, Leleux B (1996) Trade-offs in the investment decisions of European venture capitalists. J Bus Ventur 11(4):273–287. https://doi.org/10.1016/0883-9026(95)00126-3
Naous D, Legner C (2019) Understanding users’ preferences for privacy and security features—a conjoint analysis of cloud storage services. In: International conference on business information systems. Springer, Cham, pp 352–365
Nawaz F, Asadabadi MR, Janjua NK, Hussain OK, Chang E, Saberi M (2018) An MCDM method for cloud service selection using a Markov chain and the best–worst method. Knowl Based Syst 159:120–131. https://doi.org/10.1016/j.knosys.2018.06.010
Nebro AJ, Durillo JJ, Vergne M (2015) Redesigning the jMetal multi-objective optimization framework. In: Proceedings of the companion publication of the 2015 annual conference on genetic and evolutionary computation. pp 1093–1100
Oliveira GD, Dias LC (2020) The potential learning effect of a MCDA approach on consumer preferences for alternative fuel vehicles. Ann Oper Res. https://doi.org/10.1007/s10479-020-03584-x
Omrani H, Alizadeh A, Naghizadeh F (2020) Incorporating decision makers’ preferences into DEA and common weight DEA models based on the best–worst method (BWM). Soft Comput 24(6):3989–4002. https://doi.org/10.1007/s00500-019-04168-z
OpenTS. https://www.coin-or.org/Ots/docs/index.html. Accessed 2 Sep 2019
Opricovic S (1998) Multiple criteria optimization of civil engineering systems. Dissertation, University of Belgrade
Ozmen M, Aydogan EK (2020) Robust multi-criteria decision making methodology for real life logistics center location problem. Artif Intell Rev 53(1):725–751. https://doi.org/10.1007/s10462-019-09763-y
Pamučar D, Petrović I, Ćirović G (2018a) Modification of the best–worst and MABAC methods: a novel approach based on interval-valued fuzzy-rough numbers. Expert Syst Appl 91:89–106. https://doi.org/10.1016/j.eswa.2017.08.042
Pamučar D, Stević Ž, Sremac S (2018b) A new model for determining weight coefficients of criteria in mcdm models: full consistency method (FUCOM). Symmetry 10(9):393. https://doi.org/10.3390/sym10090393
Parejo JA, Racero J, Guerrero F, Kwok T, Smith KA (2003) Fom: a framework for metaheuristic optimization. In: International conference on computational science. Springer, Berlin, Heidelberg, pp 886–895
Parejo JA, Ruiz-Cortés A, Lozano S, Fernandez P (2012) Metaheuristic optimization frameworks: a survey and benchmarking. Soft Comput 16(3):527–561. https://doi.org/10.1007/s00500-011-0754-8
Parhizgarsharif A, Lork A, Telvari A (2019) A hybrid approach based on the BWM-VIKOR and GRA for ranking facility location in construction site layout for Mehr project in Tehran. Decis Sci Lett 8(3):233–248. https://doi.org/10.5267/j.dsl.2019.3.001
Piengang FCN, Beauregard Y, Kenné JP (2019) An APS software selection methodology integrating experts and decisions-maker’s opinions on selection criteria: a case study. Cogent Eng 6(1):1594509. https://doi.org/10.1080/23311916.2019.1594509
Popović M, Kuzmanović M, Martić M (2012) Using conjoint analysis to elicit employers’ preferences toward key competencies for a business manager position. In: Proceedings of the XIII international symposium symorg 2012: innovative management and business performance. pp 1258–1266
Popović M, Savić G, Kuzmanović M, Martić M (2020) Using data envelopment analysis and multi-criteria decision-making methods to evaluate teacher performance in higher education. Symmetry 12(4):563. https://doi.org/10.3390/sym12040563
Pullman ME, Moore WL, Wardell DG (2002) A comparison of quality function deployment and conjoint analysis in new product design. J Prod Innov Manag 19(5):354–364. https://doi.org/10.1111/1540-5885.1950354
R Package “Conjoint”. https://cran.r-project.org/web/packages/conjoint/conjoint.pdf. Accessed 8 Nov 2019
Ramírez A, Romero JR, García-Martínez C, Ventura S (2019) JCLEC-MO: a Java suite for solving many-objective optimization engineering problems. Eng Appl Artif Intel 81:14–28. https://doi.org/10.1016/j.engappai.2019.02.003
Rao VR (2008) Developments in conjoint analysis. In: Handbook of marketing decision models. Springer, Boston, pp 23–53
Rao VR (2014) Applied conjoint analysis. Springer, New York
Rezaei J (2015) Best–worst multi-criteria decision-making method. Omega 53:49–57. https://doi.org/10.1016/j.omega.2014.11.009
Rezaei J (2016) Best–worst multi-criteria decision-making method: some properties and a linear model. Omega 64:126–130. https://doi.org/10.1016/j.omega.2015.12.001
Rezaei J, Nispeling T, Sarkis J, Tavasszy L (2016) A supplier selection life cycle approach integrating traditional and environmental criteria using the best worst method. J Clean Prod 135:577–588. https://doi.org/10.1016/j.jclepro.2016.06.125
Rezaei J, Kothadiya O, Tavasszy L, Kroesen M (2018) Quality assessment of airline baggage handling systems using SERVQUAL and BWM. Tourism Manag 66:85–93. https://doi.org/10.1016/j.tourman.2017.11.009
Rhee HT, Yang SB (2015) Does hotel attribute importance differ by hotel? Focusing on hotel star-classifications and customers’ overall ratings. Comput Hum Behav 50:576–587. https://doi.org/10.1016/j.chb.2015.02.069
Roberge V, Tarbouchi M, Okou F (2015) gpuMF: a Framework for Parallel hybrid metaheuristics on GPU with application to the minimization of harmonics in multilevel inverters. Int J Process Syst Eng 3(1–3):20–41. https://doi.org/10.1504/IJPSE.2015.071426
Roy B (1968) Classement et choix en pr_esence de points de vue multiples (Le methode ELECTRE) 2: 57–75
Rummler A, Strufe T (2004) Evolvica—a framework for evolutionary computation. https://pdfs.semanticscholar.org/756a/6382817b12d556c901de6b5138ae07ca8a6d.pdf. Accessed 2 Sep 2019
Saaty TL (1980) The analytical hierarchy process. McGraw-Hill, New York
Sacco W, De Oliveira C (2005) A new stochastic optimization algorithm based on a particle collision metaheuristic. In: Proceedings of the 6th world congress of structural and multidisciplinary optimization, WCSMO, Rio de Janeiro, pp 1–6
Sadjadi S, Karimi M (2018) Best–worst multi-criteria decision-making method: a robust approach. Decis Sci Lett 7(4):323–340. https://doi.org/10.5267/j.dsl.2018.3.003
Safarzadeh S, Khansefid S, Rasti-Barzoki M (2018) A group multi-criteria decision-making based on best–worst method. Comput Ind Eng 126:111–121. https://doi.org/10.1016/j.cie.2018.09.011
Salimi N, Rezaei J (2018) Evaluating firms’ R&D performance using best worst method. Eval Program Plan 66:147–155. https://doi.org/10.1016/j.evalprogplan.2017.10.002
Saridakis C (2009) A hybrid conjoint model for the identification of the UK supermarket choice determinants: an exploratory study. Int Rev Retail Distrib Consum Res 19(2):103–133. https://doi.org/10.1080/09593960903109451
SAWTOOTH. https://www.sawtoothsoftware.com/products/conjoint-choice-analysis. Accessed 8 Nov 2019
Sen CG, Baracli H (2010) Fuzzy quality function deployment based methodology for acquiring enterprise software selection requirements. Expert Syst Appl 37(4):3415–3426. https://doi.org/10.1016/j.eswa.2009.10.006
Sen CG, Baracli H, Sen S, Basligil H (2009) An integrated decision support system dealing with qualitative and quantitative objectives for enterprise software selection. Expert Syst Appl 36(3):5272–5283. https://doi.org/10.1016/j.eswa.2008.06.070
Serrai W, Abdelli A, Mokdad L, Hammal Y (2017) Towards an efficient and a more accurate web service selection using MCDM methods. J Comput Sci 22:253–267. https://doi.org/10.1016/j.jocs.2017.05.024
Shepherd DA, Zacharakis A (1999) Conjoint analysis: a new methodological approach for researching the decision policies of venture capitalists. Venture Capital Int J Entrep Finance 1(3):197–217. https://doi.org/10.1080/136910699295866
Silayoi P, Speece M (2007) The importance of packaging attributes: a conjoint analysis approach. Eur J Market 41(11/12):1495–1517. https://doi.org/10.1108/03090560710821279
Silva S, Almeida J (2003) GPLAB-a genetic programming toolbox for MATLAB. In: Proceedings of the Nordic MATLAB conference. pp 273–278
Sofuoglu MA (2020) Fuzzy applications of best–worst method in manufacturing environment. Soft Comput 24(1):647–659. https://doi.org/10.1007/s00500-019-04491-5
Sörensen K, Sevaux M, Glover F (2018) A history of metaheuristics. In: Martí R, Panos P, Resende M (eds) Handbook of heuristics. Springer, Cham, pp 1–18
Sotoudeh-Anvari A, Hafezalkotob A (2018) A bibliography of metaheuristics-review from 2009 to 2015. Int J Knowl Based Intell Eng Syst 22(1):83–95. https://doi.org/10.3233/KES-180376
Soutar GN, Turner JP (2002) Students’ preferences for university: a conjoint analysis. Int J Educ Manag 16(1):40–45. https://doi.org/10.1108/09513540210415523
SPSS 24, IBM SPSS Conjoint 24 (2016) ftp://public.dhe.ibm.com/software/analytics/spss/documentation/statistics/24.0/en/client/Manuals/IBM_SPSS_Conjoint.pdf. Accessed 8 Nov 2019
STATA CHOICE MODELS. https://www.stata.com/manuals/cm.pdf. Accessed 8 Nov 2019
Subramanian N, Ramanathan R (2012) A review of applications of analytic hierarchy process in operations management. Int J Prod Econ 138(2):215–241. https://doi.org/10.1016/j.ijpe.2012.03.036
Tayarani MH, Akbarzadeh TNMR (2008) Magnetic optimization algorithms a new synthesis. In: 2008 IEEE Congress on evolutionary computation, CEC 2008. pp 2659–2664. https://doi.org/10.1109/CEC.2008.4631155
Tian Y, Cheng R, Zhang X, Jin Y (2017) PlatEMO: A MATLAB platform for evolutionary multi-objective optimization. IEEE Comput Intell Mag 12(4):73–87. https://doi.org/10.1109/MCI.2017.2742868
Tonda A (2019) Inspyred: bio-inspired algorithms in python. Genet Program Evol Mach. https://doi.org/10.1007/s10710-019-09367-z
Vafadarnikjoo A, Tavana M, Botelho T, Chalvatzis K (2020) A neutrosophic enhanced best–worst method for considering decision-makers’ confidence in the best and worst criteria. Ann Oper Res. https://doi.org/10.1007/s10479-020-03603-x
Ventura S, Romero C, Zafra A, Delgado JA, Hervás C (2008) JCLEC: a Java framework for evolutionary computation. Soft Comput 12(4):381–392. https://doi.org/10.1007/s00500-007-0172-0
Voudouris C, Dorne R, Lesaint D, Liret A (2001) iOpt: a software toolkit for heuristic search methods. In: International conference on principles and practice of constraint programming. Springer, Berlin, Heidelberg, pp 716–729
Vrbančič G, Brezočnik L, Mlakar U, Fister D, Fister I (2018) NiaPy: Python microframework for building nature-inspired algorithms. J Open Source Softw 3(23):613. https://doi.org/10.21105/joss.00613
Wagner S (2009) Heuristic optimization software systems modeling of heuristic optimization algorithms in the heuristic lab software environment. Dissertation, Johannes Kepler University
Wagner S, Beham A, Kronberger G, Kommenda M, Pitzer E, Kofler M, et al. (2010) HeuristicLab 3.3: a unified approach to metaheuristic optimization. In: Actas del séptimo congreso español sobre Metaheurísticas, Algoritmos Evolutivos y Bioinspirados
Wang X (2019) Health service design with conjoint optimization. J Oper Res Soc 70(7):1091–1101. https://doi.org/10.1080/01605682.2018.1489341
White DR (2012) Software review: the ECJ toolkit. Genet Program Evol Mach 13(1):65–67. https://doi.org/10.1007/s10710-011-9148-z
William E, Northern J (2008) Genetic programming lab (GPLab) tool set version 3.0. In: 2008 IEEE region 5 conference. IEEE, pp 1–6
Wu WY, Liao YK, Chatwuthikrai A (2014) Applying conjoint analysis to evaluate consumer preferences toward subcompact cars. Expert Syst Appl 41(6):2782–2792. https://doi.org/10.1016/j.eswa.2013.10.011
Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: González JR, Pelta DA, Cruz C, Terrazas G, Krasnogor N (eds) Nature inspired cooperative strategies for optimization. Springer, Berlin, pp 65–74. https://doi.org/10.1007/978-3-642-12538-6_6
Yazgan HR, Boran S, Goztepe K (2009) An ERP software selection process using artificial neural network based on analytic network process approach. Expert Syst Appl 36(5):9214–9222. https://doi.org/10.1016/j.eswa.2008.12.022
Yeap JA, Ignatius J, Ramayah T (2014) Determining consumers’ most preferred eWOM platform for movie reviews: a fuzzy analytic hierarchy process approach. Comput Hum Behav 31:250–258. https://doi.org/10.1016/j.chb.2013.10.034
Zaidan AA, Zaidan BB, Hussain M, Haiqi A, Kiah MM, Abdulnabi M (2015) Multi-criteria analysis for OS-EMR software selection problem: a comparative study. Decis Support Syst 78:15–27. https://doi.org/10.1016/j.dss.2015.07.002
Zandi F, Tavana M (2012) A fuzzy group multi-criteria enterprise architecture framework selection model. Expert Syst Appl 39(1):1165–1173. https://doi.org/10.1016/j.eswa.2011.07.120
Zavadskas EK, Turskis Z (2010) A new additive ratio assessment (ARAS) method in multi-criteria decision-making. Technol Econ Dev Econ 16(2):159–172. https://doi.org/10.3846/tede.2010.10
Zavadskas EK, Kaklauskas A, Sarka V (1994) The new method of multicriteria complex proportional assessment of projects. Technol Econ Develop Econ 1(3):131–139
Zavadskas EK, Turskis Z, Kildiene S (2014) State of art surveys of overviews on MCDM/MADM methods. Technol Econ Dev Econ 20(1):165–179. https://doi.org/10.3846/20294913.2014.892037
Acknowledgments
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. We would like to thank the 14 experts for their valuable contribution to the study. We are also appreciated to the framework developers (Aurora Ramírez Quesada, Aaron Garrett, Grega Vrbančič, Antonio Benítez Hidalgo, Herman De Beukelaer, François-Michel De Rainville, Sebastian Ventura, Xingyi Zhang, Zoltan Mann, Cristian Lang, Christian Gagné) who do not hesitate to answer our questions about their frameworks.
Funding
The authors declare that there is no financial support for this research.
Author information
Authors and Affiliations
Contributions
Investigation, software, writing—original Draft, writing—review and editing were performed by GZÖ. Conceptualization, methodology, supervision, writing—review and editing were performed by SE. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
Author Gülin Zeynep Öztaş declares that she has no conflict of interest. Author Sabri Erdem declares that he/she has no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Only the abstract was submitted to the “20th International Symposium on Econometrics, Operations Research and Statistics” for oral presentation.
Appendix 1
Rights and permissions
About this article
Cite this article
Oztas, G.Z., Erdem, S. Framework selection for developing optimization algorithms: assessing preferences by conjoint analysis and best–worst method. Soft Comput 25, 3831–3848 (2021). https://doi.org/10.1007/s00500-020-05411-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-05411-8