Abstract
Multiple criteria sorting methods assign objects into ordered categories while objects are characterized by a vector of n attributes values. Categories are ordered, and the assignment of the object is monotonic w.r.t. to some underlying order on the attributes scales (criteria). We drew a landscape of these methods in Part I “Survey of the literature” (published in a previous issue of the present journal) and we aim to provide a theoretical view of the field in this second part. We describe a general framework for MCS models and position some existing models in the picture. Issues related to imperfect or insufficient information are then discussed. We also address questions that arise in the final phase of a decision aiding process as, e.g., explaining a recommendation or suggesting efficient ways of improving an object assignment.

Similar content being viewed by others
Notes
The main model of this type is the Choquet integral model (see, e.g., Grabisch 2016, Chapter 6). This model has been characterized by Wakker (1989, Theorem VI.5.1) in a ranking context. However, the latter result applies only in the case all criteria are evaluated on a common scale. This is a convenient setting for decision under uncertainty, but raises difficulties related to criteria scales commensurateness in a MCDM/A context (Grabisch 2016, p. 343).
Different implementations of an “as well as possible” requirement can be found in the literature. Usually, slack variables are introduced in the linear programming formulation. There are several ways of doing so (see, e.g., Jacquet-Lagrèze 1982; Doumpos and Zopounidis 2002). The objective function of the LP minimizes the sum of the slack variables, or the maximal value of slack variables, or a linear combination of such objectives. A positive slack variable means that a constraint is violated. Minimizing the number of wrong assignments requires a MIP formulation.
Note that the latter term is classical in this sense in the field of classification (Aggarwal et al. 2010) but may be ambiguous because some authors use it designate the learning of a sorting model from assignment examples.
References
Aggarwal CC, Chen C, Han J (2010) The inverse classification problem. J Comput Sci Technol 25(3):458–468
Almeida-Dias J, Figueira JR, Roy B (2010) ELECTRE TRI-C: a multiple criteria sorting method based on characteristic reference actions. Eur J Oper Res 204(3):565–580
Bazin A, Couceiro M, Devignes M-D, Napoli A (2020) Explaining multicriteria decision making with formal concept analysis. In: Concept lattices and applications 2020, volume 2668 of CEUR Workshop Proceedings, pp 119–130
Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2017) Explaining robust additive utility models by sequences of preference swaps. Theor Decis 82(2):151–183
Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2017) A model for accountable ordinal sorting. In: Carles Sierra, editor, proceedings of the twenty-sixth international joint conference on artificial intelligence, IJCAI 2017, Melbourne, Australia, Aug 19-25, 2017, pp 814–820
Belahcène K, Labreuche C, Maudet N, Mousseau V, Ouerdane W (2018) Accountable approval sorting. In: Proceedings of the 27th international joint conference on artificial intelligence (IJCAI 2018)
Belahcène K, Mousseau V, Ouerdane W, Pirlot M, Sobrie O (2022) Multiple criteria sorting models and methods. Part I: Survey of the literature. 4OR. https://doi.org/10.1007/s10288-022-00530-4 (to appear)
Bouyssou D, Marchant T (2007) An axiomatic approach to noncompensatory sorting methods in MCDM, I: the case of two categories. Eur J Oper Res 178(1):217–245
Bouyssou D, Marchant T (2007) An axiomatic approach to noncompensatory sorting methods in MCDM, II: more than two categories. Eur J Oper Res 178(1):246–276
Bouyssou D, Marchant T (2010) Additive conjoint measurement with ordered categories. Eur J Oper Res 203(1):195–204
Bouyssou D, Marchant T (2015) On the relations between ELECTRE TRI-B and ELECTRE TRI-C and on a new variant of ELECTRE TRI-B. Eur J Oper Res 242(1):201–211
Bouyssou D, Marchant T, Pirlot M (2009) A conjoint measurement approach to the discrete Sugeno integral. Essays in Honor of Peter C. In: Brams S, Gehrlein WV, Roberts FS (eds) The Mathematics of Preference, Choice and Order. Springer, Fishburn, pp 85–109
Bouyssou D, Marchant T, Pirlot M (2021) A note on ELECTRE TRI-nB with few limiting profiles. 4OR
Bouyssou D, Marchant T, Pirlot M (2022) A theoretical look at Electre Tri-nB and related sorting models. 4OR
Doshi-Velez F, Kortz M, Budish R, Bavitz Ch, Gershman S, O’Brien D, Schieber S, Waldo J, Weinberger D, Wood A (2017) Accountability of AI under the law: The role of explanation. CoRR, arXiv:1711.01134
Doumpos M, Zopounidis C (2002) Multicriteria decision aid classification methods. Kluwer Academic Publishers, Dordrecht Boston
Dubois D, Marichal J-L, Prade H, Roubens M, Sabbadin R (2001) The use of the discrete Sugeno integral in decision making: a survey. Internat J Uncertain Fuzziness Knowledge-Based Systems 9:539–561
Fernández E, Figueira JR, Navarro J, Roy B (2017) ELECTRE TRI-nB: a new multiple criteria ordinal classification method. Eur J Oper Res 263(1):214–224
Fernández E, Figueira JR, Navarro J (2019) An interval extension of the outranking approach and its application to multiple-criteria ordinal classification. Omega 84:189–198
Fernández E, Figueira JR, Navarro J (2020) Interval-based extensions of two outranking methods for multi-criteria ordinal classification. Omega 95:102065
Fürnkranz J, Hüllermeier E (2010) Preference learning: an introduction. In: J. Fürnkranz and E. Hüllermeier, (eds), Preference Learning, Springer pp 1–17
Ghahraman A, Prior D (2016) A learning ladder toward efficiency: proposing network-based stepwise benchmark selection. Omega 63:83–93
Goldstein WM (1991) Decomposable threshold models. J Math Psychol 35(1):64–79
Grabisch M (2016) Remarkable polyhedra related to set functions, games and capacities. TOP 24:301–326
Greco S, Matarazzo B, Słowiński R (2001) Conjoint measurement and rough set approach for multicriteria sorting problems in presence of ordinal criteria. In: Colorni A, Paruccini M, Roy B (eds) A-MCD-A. aide multicritère à la Décision / multiple criteria decision aid. European Commission, Joint Research Centre, Luxembourg, pp 117–144
Greco S, Mousseau V, Słowiński R (2010) Multiple criteria sorting with a set of additive value functions. Eur J Oper Res 207(3):1455–1470
Greco S, Mousseau V, Słowiński R (2014) Robust ordinal regression for value functions handling interacting criteria. Eur J Oper Res 239(3):711–730
Gunning D, Aha D (2019) DARPA’s explainable artificial intelligence (XAI) program. AI Mag 40(2):44–58
Guo M, Liao X, Liu J (2019) A progressive sorting approach for multiple criteria decision aiding in the presence of non-monotonic preferences. Expert Systems and Applications 123:1–17
Jacquet-Lagrèze E (1982) Binary preference indices: a new look on multicriteria aggregation procedures. Eur J Oper Res 10(1):25–32
Jacquet-Lagrèze E, Siskos Y (1982) Assessing a set of additive utility functions for multicriteria decision making: the UTA method. Eur J Oper Res 10:151–164
Janssen P, Nemery P (2013) An extension of the flowsort sorting method to deal with imprecision. 4OR 11:171–193
Kadziński M, Ciomek K (2016) Integrated framework for preference modeling and robustness analysis for outranking-based multiple criteria sorting with ELECTRE and PROMETHEE. Inf Sci 352–353:167–187
Kadziński M, Ciomek K, Rychly P, Słowiński R (2016) Post factum analysis for robust multiple criteria ranking and sorting. J Global Optim 65(3):531–562
Kadziński M, Ghaderi M, Dąbrowski M (2020) Contingent preference disaggregation model for multiple criteria sorting problem. Eur J Oper Res 281(2):369–387
Kadziński M, Stamenković M, Uniejewski M (2022) Stepwise benchmarking for multiple criteria sorting. Omega 108:102579
Köksalan M, Mousseau V, Özpeynirci Ö, Özpeynirci SB (2009) A new outranking-based approach for assigning alternatives to ordered classes. Nav Res Logistics 56:74–85
Labreuche C (2011) A general framework for explaining the results of a multi-attribute preference model. Artif Intell 175(7):1410–1448
Labreuche C, Fossier S (2018) Explaining multi-criteria decision aiding models with an extended shapley value. In: IJCAI International joint conference on artificial intelligence, pp 331 - 339
Labreuche C, Maudet N, Ouerdane W (2011) Minimal and complete explanations for critical multi-attribute decisions. In: Algorithmic decision theory, pp 121–134
Labreuche C, Maudet N, Ouerdane W (2012) Justifying dominating options when preferential information is incomplete. In: Proceedings of the 20th European conference on artificial intelligence, ECAI’12, pp 486-491. IOS Press
Liu J, Kadziński M, Liao X, Mao X, Wang Y (2020) A preference learning framework for multiple criteria sorting with diverse additive value models and valued assignment examples. Eur J Oper Res 286(3):963–985
Meyer P, Olteanu A-L (2019) Handling imprecise and missing evaluations in multi-criteria majority-rule sorting. Comput Oper Res 110:135–147
Minoungou P, Mousseau V, Ouerdane W, Scotton P (2022) A MIP-based approach to learn MR-Sort models with single-peaked preferences. Ann Oper Res
Mousseau V, Figueira JR, Dias LC, Gomes da Silva C, Clímaco J (2003) Resolving inconsistencies among constraints on the parameters of an MCDA model. Eur J Oper Res 147(1):72–93
Mousseau V, Dias LC, Figueira J (2006) Dealing with inconsistent judgments in multiple criteria sorting models. 4OR 4:145–158
Mousseau V, Özpeynirci Ö, Özpeynirci S (2018) Inverse multiple criteria sorting problem. Ann Oper Res 267(1–2):379–412
Pelissari R, Oliveira MC, Abackerli AJ, Ben Amor S, Pontes Assumpção MR (2021) Techniques to model uncertain input data of multi-criteria decision-making problems: a literature review. Int Trans Oper Res 28(2):523–559
Pereira J, de Oliveira ECB, Gomes LFAM, Araujo RM (2019) Sorting retail locations in a large urban city by using ELECTRE TRI-C and trapezoidal fuzzy numbers. Soft Comput 23(12):4193–4206
Pereira J, de Oliveira ECB, Morais DC, Costa APCS, López PA (2019) ELECTRE TRI-C with hesitant outranking functions: application to supplier development. J Intell Fuzzy Syst 37(6):7923–7933
Petrović M, Bojković N, Anić I, Stamenković M, Pejčić-Tarle S (2014) An ELECTRE-based decision aid tool for stepwise benchmarking: anapplication over EU Digital Agenda targets. Decis Support Syst 59:230–241
Rocha C, Dias LC (2008) An algorithm for ordinal sorting based on ELECTRE with categories defined by examples. J Global Optim 42(2):255–277
Roy B, Bouyssou D (1993) Aide multicritère à la décision: méthodes et cas. Economica Paris
Słowínski R, Greco S, Matarazzo B (2002) Axiomatization of utility, outranking and decision-rule preference models for multiple-criteria classification problems under partial inconsistency with the dominance principle. Control Cybern 31(4):1005–1035
Tehrani AF, Hüllermeier E (2013) Ordinal Choquistic regression. In: Proceedings of the 8th conference of the European society for fuzzy logic and technology. Atlantis Press
Tehrani AF, Cheng W, Dembczyński K, Hüllermeier E (2012) Learning monotone nonlinear models using the Choquet integral. Mach Learn 89(1–2):183–211
Wakker PP (1989) Additive representations of preferences: a new foundation of decision analysis. Kluwer, Dordrecht
Walton D (1996) Argumentation schemes for presumptive reasoning. Erlbaum, Mahwah
Wei Y (1992) Aide multicritère à la décision dans le cadre de la problématique du tri : concepts, méthodes et applications. Thèse de doctorat, Université Paris Dauphine, Paris, France (in French)
Zhong Q, Fan X, Luo X, Toni F (2019) An explainable multi-attribute decision model based on argumentation. Expert Syst Appl 117:42–61
Acknowledgements
We are grateful to Denis Bouyssou for reading a previous version of the manuscript and making a number of relevant comments. We also thank the Editors for inviting us to write this survey and for their observations on the final draft. Of course, the responsibility for errors and omissions in this paper as well as the opinions that are expressed remains entirely with the authors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A list of abbreviations
Appendix A list of abbreviations
For the reader’s convenience, we list below, in alphabetic order, the acronyms used in the text, except for acronyms of sorting methods.
-
AVF: Additive Value Function
-
CAI: Class Acceptability Index
-
DM: Decision Maker
-
DRSA: Dominance based Rough Sets Approach
-
LP: Linear Program
-
MCDM/A: Multiple Criteria Decision Making / Aiding
-
MCS: Multiple Criteria Sorting
-
MILP: Mixed Integer Linear Program
-
ML: Machine Learning
-
MOP: Monotone Ordered Partition
-
PL: Preference Learning
-
ROR: Robust Ordinal Regression
-
SMAA: Stochatic Multicriteria Acceptability Analysis
-
VF: Value Function
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Belahcène, K., Mousseau, V., Ouerdane, W. et al. Multiple criteria sorting models and methods. Part II: theoretical results and general issues. 4OR-Q J Oper Res 21, 181–204 (2023). https://doi.org/10.1007/s10288-022-00531-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10288-022-00531-3
Keywords
- Multiple criteria decision making
- Multiple criteria sorting
- Monotone classification
- Preference learning