Skip to main content
Springer Nature Link
Log in

Adapting Thurstone’s Law of Comparative Judgment to fuse preference orderings in manufacturing applications

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

A rather common problem in the manufacturing field includes: (i) a collection of objects to be compared on the basis of the degree of some attribute, (ii) a set of judges that individually express their subjective judgments on these objects, and (iii) a single collective judgment, which is obtained by fusing the previous subjective judgments. The goal of this contribution is to develop a new technique that combines the Thurstone’s Law of Comparative Judgment with an ad hoc response mode based on preference orderings. Apart from being relatively practical and user-friendly, this technique allows to express the collective judgment of objects on a ratio scale and is applicable to a variety of practical contexts in the field of manufacturing. The description of the proposed technique is integrated with the application to a practical case study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. For simplicity, we consider single-ended psychological continua, in which the objects’ attribute progresses in one direction only, starting from the absolute-zero point; this is reasonable when the attribute has a positive connotation exclusively (e.g., the importance of a set of product requirements) (Torgerson 1958).

  2. This assumption is quite common for psychometric studies on subjective perceptions (Torgerson 1958; Lim 2011).

  3. According to the terminology introduced in Franceschini et al. (2007), the term “preference”—defined as subjective and non-empirical (i.e., which does not necessarily stem from a direct observation of reality) assignment of numbers/symbols to properties of objects—should be replaced with the term “evaluation”—defined as subjective and empirical (i.e., which stems from a direct observation of reality) assignment. Despite this, for the sake of simplicity the term “preference” will be hereafter used.

References

  • Alwin, D. F., & Krosnick, J. A. (1985). The measurement of values in surveys: A comparison of ratings and rankings. Public Opinion Quarterly,49(4), 535–552.

    Article  Google Scholar 

  • Andrich, D. (1978). Relationships between the Thurstone and Rasch approaches to item scaling. Applied Psychological Measurement,2(3), 451–462.

    Article  Google Scholar 

  • Çakır, S. (2018). An integrated approach to machine selection problem using fuzzy SMART-fuzzy weighted axiomatic design. Journal of Intelligent Manufacturing,29(7), 1433–1445.

    Article  Google Scholar 

  • Chen, L. H., Ko, W. C., & Yeh, F. T. (2017). Approach based on fuzzy goal programing and quality function deployment for new product planning. European Journal of Operational Research,259(2), 654–663.

    Article  Google Scholar 

  • De Battisti, F., Nicolini, G., & Salini, S. (2010). The Rasch model in customer satisfaction survey data. Quality Technology & Quantitative Management,7(1), 15–34.

    Article  Google Scholar 

  • Den Ouden, E., Yuan, L., Sonnemans, P. J., & Brombacher, A. C. (2006). Quality and reliability problems from a consumer’s perspective: An increasing problem overlooked by businesses? Quality and Reliability Engineering International,22(7), 821–838.

    Article  Google Scholar 

  • DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). London: Sage.

    Google Scholar 

  • Edwards, A. L. (1957). Techniques of attitude scale construction. New York: Irvington Publishers.

    Book  Google Scholar 

  • Fishburn, P. C. (1973). The theory of social choice. Princeton: Princeton University Press.

    Google Scholar 

  • Franceschini, F., Galetto, M., & Maisano, D. (2007). Management by measurement: Designing key indicators and performance measurement systems. Berlin: Springer.

    Google Scholar 

  • Franceschini, F., Galetto, M., & Maisano, D. (2019). Designing performance measurement systems: Theory and practice of key performance indicators., Management for professionals Berlin: Springer.

    Book  Google Scholar 

  • Franceschini, F., & Maisano, D. (2015). Prioritization of QFD customer requirements based on the law of comparative judgments. Quality Engineering,27(4), 437–449.

    Article  Google Scholar 

  • Franceschini, F., & Maisano, D. (2018). Fusion of partial orderings for decision problems in quality management. In Proceedings of the 3rd international conference on quality engineering and management (ICQEM 2018), July 11–13, 2018, Barcelona (Spain).

  • Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2016). A new proposal for fusing individual preference orderings by rank-ordered agents: A generalization of the Yager’s algorithm. European Journal of Operational Research,249(1), 209–223.

    Article  Google Scholar 

  • Gulliksen, H. (1956). A least squares solution for paired comparisons with incomplete data. Psychometrika,21, 125–134.

    Article  Google Scholar 

  • Harzing, A. W., Baldueza, J., Barner-Rasmussen, W., Barzantny, C., Canabal, A., Davila, A., et al. (2009). Rating versus ranking: What is the best way to reduce response and language bias in cross-national research? International Business Review,18(4), 417–432.

    Article  Google Scholar 

  • Hosseini, S., & Al Khaled, A. (2016). A hybrid ensemble and AHP approach for resilient supplier selection. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-016-1241-y.

    Article  Google Scholar 

  • Keeney, R. L., & Raiffa, H. (1993). Decisions with multiple objectives: Preferences and value trade-offs. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Krynicki, J. C. (2006). Introduction to soft metrology. In XVIII IMEKO World Congress, 17–22 September, Rio de Janeiro (Brazil).

  • Lim, J. (2011). Hedonic scaling: A review of methods and theory. Food Quality and Preference,22(8), 733–747.

    Google Scholar 

  • Lin, C. J., & Cheng, L. Y. (2017). Product attributes and user experience design: How to convey product information through user-centered service. Journal of Intelligent Manufacturing,28(7), 1743–1754.

    Article  Google Scholar 

  • Maier, J. R. A., & Fadel, G. M. (2007). A taxonomy and decision support for the design and manufacture of types of product families. Journal of Intelligent Manufacturing,18(1), 31–45.

    Article  Google Scholar 

  • Morrissey, J. H. (1955). New method for the assignment of psychometric scale values from incomplete paired comparisons. JOSA,45(5), 373–378.

    Article  Google Scholar 

  • Nederpelt, R., & Kamareddine, F. (2004). Logical reasoning: A first course. London: King’s College Publications.

    Google Scholar 

  • Ngan, T. T., Tuan, T. M., Son, L. H., Minh, N. H., & Dey, N. (2016). Decision making based on fuzzy aggregation operators for medical diagnosis from dental X-ray images. Journal of Medical Systems,40(12), 280.

    Article  Google Scholar 

  • Önüt, S., Kara, S. S., & Efendigil, T. (2008). A hybrid fuzzy MCDM approach to machine tool selection. Journal of Intelligent Manufacturing,19(4), 443–453.

    Article  Google Scholar 

  • Paruolo, P., Saisana, M., & Saltelli, A. (2013). Ratings and rankings: Voodoo or science? Journal of the Royal Statistical Society: Series A (Statistics in Society),176(3), 609–634.

    Article  Google Scholar 

  • Qazi, A., Quigley, J., Dickson, A., & Ekici, Ş. Ö. (2017). Exploring dependency based probabilistic supply chain risk measures for prioritising interdependent risks and strategies. European Journal of Operational Research,259(1), 189–204.

    Article  Google Scholar 

  • Roberts, F. S. (1979). Measurement theory: With applications to decisionmaking, utility, and the social sciences (Vol. 7)., Encyclopedia of mathematics and its applications Reading, MA: Addison-Wesley.

    Google Scholar 

  • Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences,1(1), 83–98.

    Article  Google Scholar 

  • Stevens, S. S. (1946). On the theory of scales of measurement. Science,103, 2684.

    Article  Google Scholar 

  • Tao, F., Zhang, L., & Laili, Y. (2016). Configurable intelligent optimization algorithm. Berlin: Springer. ISBN 978-3-319-08839-6.

    Google Scholar 

  • Tarricone, P., & Newhouse, C. P. (2016). Using comparative judgement and online technologies in the assessment and measurement of creative performance and capability. International Journal of Educational Technology in Higher Education,13(1), 16.

    Article  Google Scholar 

  • Thurstone, L. L. (1927). A law of comparative judgments. Psychological Review,34(4), 273.

    Article  Google Scholar 

  • Thurstone, L. L., & Jones, L. V. (1957). The rational origin for measuring subjective values. Journal of the American Statistical Association,52(280), 458–471.

    Article  Google Scholar 

  • Torgerson, W. S. (1958). Theory and methods of scaling. Oxford: Wiley.

    Google Scholar 

  • Trochim, W., Donnelly, J. P., & Arora, K. (2016). Research methods: The essential knowledge base (2nd ed.). Boston: Cengage. ISBN 978-1-133-95477-4.

    Google Scholar 

  • Van Kleef, E., Van Trijp, H. C., & Luning, P. (2005). Consumer research in the early stages of new product development: A critical review of methods and techniques. Food Quality and Preference,16(3), 181–201.

    Article  Google Scholar 

  • Vasquez-Espinosa, R. E., & Conners, R. W. (1982). The law of comparative judgment: Theory and implementation (No. RSIP/TR-403.82). Louisiana State, University of Baton Rouge (Remote Sensing and Image Processing Lab).

  • Vora, A., Paunwala, C. N., & Paunwala, M. (2014). Improved weight assignment approach for multimodal fusion. In Proceedings of the 2014 international conference on circuits, systems, communication and information technology applications (CSCITA 2014) (pp. 70–74).

  • Westland, S., Li, Y., & Cheung, V. (2014). Monte Carlo analysis of incomplete paired-comparison experiments. Journal of Imaging Science and Technology,58(5), 50506.1–50506.6.

    Article  Google Scholar 

  • Zeshui, X. (2012). Linguistic decision making: Theory and methods. Berlin: Springer. ISBN 978-3-642-29440-2.

    Google Scholar 

  • Zheng, P., Xu, X., & Xie, S. Q. (2016). A weighted interval rough number based method to determine relative importance ratings of customer requirements in QFD product planning. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-016-1224-z.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management and Production Engineering (DIGEP), Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129, Turin, Italy

    F. Franceschini & D. Maisano

Authors
  1. F. Franceschini
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. D. Maisano
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to F. Franceschini.

Appendix

Appendix

A.1 Torgerson’s anchoring

This section exemplifies the anchoring technique by Torgerson (1958, p. 196), applying it to the LCJ scaling in Fig. 11. Focussing on this scaling process, it can be seen that the (input) paired-comparison relationships are identical to those in the example in Fig. 8, except that those with at least one of the dummy/anchor objects are not present. The resulting (non-anchored) scale is reported in Fig. 11d.

Fig. 11
figure 11

Example of application of the LCJ, considering the paired-comparison relationships by five judges (J1 to J5) on four objects (O1 to O4). These relationships are identical to those in the example in Fig. 9, except that those with at least one of the dummy/anchor objects are not present

Full size image

The rationale of the Torgerson’s anchoring is that results of the LCJ are (at least roughly) correlated with those resulting from the so-called Method of Single Stimuli, in which each judge directly assigns the objects’ scale values, with respect to two anchors: (1) a (presumed) absolute zero, corresponding to the absence of the attribute, and (2) the maximum-imaginable degree of the attribute, conventionally set to 5. While aware of the difficulty and potential roughness of these direct assignments, Torgerson (1958, p. 196) suggests their use just for the purpose of anchoring the LCJ scale.

Subsequently, judge assignments are aggregated—object by object—through a central tendency indicator, such as the mean or median value (g), and plot against the scale values (x) computed from the LCJ. Then, a straight line to the points is fitted and the intercept on the horizontal axis (g = 0) is taken as estimate of the position of the absolute-zero point (Z) and that on the horizontal line (g = 5) as estimate of the position of the point with maximum-imaginable degree (M) of the attribute.

Considering the example in Fig. 11, we hypothesize that the five judges directly assign the objects’ scale values on a rating scale from 0 to 5, with unitary resolution; the zero point corresponds to the absence while the maximum value (i.e., 5) corresponds to the maximum-imaginable degree of the attribute. Table 3 collects these assignments.

Table 3 Direct assignments of the scale values for six objects (i.e., O1 to O4, Z and M), by five judges (J1 to J5)
Full size table

Assignments are then aggregated using the arithmetic mean. The graph in Fig. 12 plots the resulting mean values (g) against the scale values (x) obtained through the LCJ (see Fig. 11). Then, a straight tendency line is fitted (through a linear least-squares regression) and the intersection of this line with the horizontal axis (g = 0) determines an estimates of the absolute-zero point (Z, i.e., first anchor), while that with the horizontal line g = 5 determines an estimate of the point (M, i.e., second anchor) of the maximum-imaginable degree of the attribute on the Thurstone’s scale. Next, the LCJ scale values are normalized in the conventional range [0, 100], through the linear transformation in Eq. 4. This scale can reasonably be considered as a ratio one (see Table 4).

Fig. 12
figure 12

Comparison of the scale values resulting from the Thurstone’s LCJ and those resulting from a direct scale-value assignment (Method of Single Stimuli) for four objects (O1 to O4)

Full size image
Table 4 Anchoring of the LCJ scale (in Fig. 11d), applying the technique by Torgerson
Full size table

We have verified that the new anchoring technique (presented in "Anchoring the Thurstone’s Scaling: the ZM-technique" section) provides results in line with those obtained from the Torgerson’s technique. e.g., Fig. 13 shows that these two anchoring techniques, when applied to the same scaling problem, are strongly correlated. Also, we have empirically observed that the correlation tends to increase for problems with a larger number of objects and/or judges.

Fig. 13
figure 13

Comparison between two anchoring techniques (i.e., that by Torgerson, exemplified in Table 4, and the proposed technique, exemplified in Fig. 9), with reference to the same LCJ-scaling problem

Full size image

A.2 Questionnaire

Figure 14 reports an example of questionnaire to guide the construction of preference orderings.

Fig. 14
figure 14

Example of questionnaire with guidance for the formulation of preference orderings

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Franceschini, F., Maisano, D. Adapting Thurstone’s Law of Comparative Judgment to fuse preference orderings in manufacturing applications. J Intell Manuf 31, 387–402 (2020). https://doi.org/10.1007/s10845-018-1452-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-018-1452-5

Keywords