In this paper, we propose an alternative novel method based on the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) to solve the problem of ranking and comparing algorithms. In evolutionary computation, algorithms are executed several times and then a statistic in terms of mean values and standard deviations are calculated. In order to compare algorithms performance it is very common to handle such issue by means of statistical tests. Ranking algorithms, e.g., by means of Friedman test may also present limitations since they consider only the mean value and not the standard deviation of the results. Since the TOPSIS is not able to handle directly this kind of data, we develop an approach based on TOPSIS for algorithm ranking named as A-TOPSIS. In this case, the alternatives consist of the algorithms and the criteria are the benchmarks. The rating of the alternatives with respect to the criteria are expressed by means of a decision matrix in terms of mean values and standard deviations. A case study is used to illustrate the method for evolutionary algorithms. The simulation results show the feasibility of the A-TOPSIS to find out the ranking of algorithms under evaluation.
