Abstract
We investigate the connection between weights, scales, and the importance of criteria, when a linear value function is assumed to be a suitable representation of a decision maker’s preferences. Our considerations are based on a simple two-criteria experiment, where the participants were asked to indicate which of the criteria was more important, and to pairwise compare a number of alternatives. We use the participants’ pairwise choices to estimate the weights for the criteria in such a way that the linear value function explains the choices to the extent possible. More specifically, we study two research questions: (1) is it possible to find a general scaling principle that makes the rank order of the importance of criteria consistent with the rank order of the magnitudes of the weights, and (2) how good is a simple, direct method of asking the decision maker to “provide” weights for the criteria compared to our estimation procedure. Our results imply that there is reason to question two common beliefs, namely that the values of the weights would reflect the importance of criteria, and that people could reliably “provide” such weights without estimation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
“Announced importance of criteria” simply means decision maker’s statements such as “criterion A is more important than criterion B”.
A more general preference model is described in Greco et al. (2001).
If X i dominates X j ,i,j∈N⇒(X i ,X j )∈P.
Dominance relations can be ignored.
For more details on “Non-Archimedean”, see Arnold et al. (1998).
Currently Aalto University, School of Business.
It would have been difficult to use participants who did not respond to all questions, because we needed the first ten responses to estimate the weights; the remaining responses were predicted.
We adopted this useful scale from Saaty (1980).
By “optimal weights” we mean weights implied by an “optimal” scale.
By “psychological dependence” we understand the situation, when the criteria are perceived to be similar, in other words to belong to the same class of criteria. For example, different recreational outdoor activity possibilities, such as fishing, hunting, etc., are understood to belong to the same class of criteria.
References
Arnold, V., Bardhan, I., Cooper, W. W., & Gallegos, A. (1998). Primal and dual optimality in computer codes using two-stage solution procedures in DEA. In J. E. Aronson & S. Zionts (Eds.), Operations research: models, methods and applications, Westport/Norwell: Quorum Books.
Choo, E., Schoner, B., & Wedley, W. C. (1999). Interpretation of criteria weights in multicriteria decision making. Computers & Industrial Engineering, 37, 527–541.
Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. The American Psychologist, 34(7), 571–582.
Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.
Fischer, G. W. (1995). Range sensitivity of attribute weights in multiattribute value model. Organizational Behavior & Human Decision Processes, 62(30), 252–266.
Goldstein, W. M. (1990). Judgments of relative importance in decision making: Global vs local interpretations of subjective weight. Organizational Behavior & Human Decision Processes, 47(2), 313–336.
Greco, S., Matarazzo, B., & Slowinski, R. (2001). Rough sets theory for multi-criteria decision analysis. European Journal of Operational Research, 129, 1–47.
Keeney, R. L., & Raiffa, H. (1976). Decisions with multiple objectives: preference and value trade-offs. New York: Wiley.
Korhonen, P., Silvennoinen, K., Wallenius, J., & Öörni, A. (2012). Can a linear value function explain choices? An experimental study. European Journal of Operational Research, 219(2), 360–367.
Podinovski, V. V. (2002). The quantitative importance of criteria for MCDA. Journal of Multi-Criteria Decision Analysis, 11, 1–15.
Podinovski, V. V. (2005). The quantitative importance of criteria with a continuous first-order metric scale. Automation and Remote Control, 66(9), 1478–1485.
Roy, B., & Mousseau, V. (1996). A theoretical framework for analysing the notion of relative importance of criteria. Journal of Multi-Criteria Decision Analysis, 5, 145–159.
Saaty, T. (1980). The analytic hierarchy process. New York: McGraw-Hill.
Salminen, P., Korhonen, P., & Wallenius, J. (1989). Testing the form of a decision maker’s multiattribute value function based on pairwise preference information. Journal of the Operational Research Society, 40(3), 299–302.
Salo, A. A., & Hämäläinen, R. P. (1992). Preference assessment by imprecise ratio statements. Operations Research, 40, 1053–1061.
Von Nitzsch, R., & Weber, M. (1993). The effect of attribute ranges on weights in multiattribute utility measurements. Management Science, 39(8), 937–943.
Zionts, S., & Wallenius, J. (1976). An interactive programming method for solving the multiple criteria problem. Management Science, 22, 652–663.
Author information
Authors and Affiliations
Corresponding author
Additional information
The research was supported by the Academy of Finland (Grant numbers 133387 and 253583).
Rights and permissions
About this article
Cite this article
Korhonen, P.J., Silvennoinen, K., Wallenius, J. et al. A careful look at the importance of criteria and weights. Ann Oper Res 211, 565–578 (2013). https://doi.org/10.1007/s10479-012-1307-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-012-1307-y