Data-driven selection of multi-criteria decision-making methods and its application to diagnosis of thyroid nodules

https://doi.org/10.1016/j.cie.2020.106490Get rights and content

Highlights

  • Data-driven selection of multi-criteria decision making (MCDM) methods is proposed.

  • Three ways are developed to compare the performance of three MCDM methods.

  • Interval numbers, fuzzy linguistic numbers, and belief distributions are unified.

  • Three ways are used to compare three methods based on diagnosis of thyroid nodules.

  • Relation between comparison results and three ways is analyzed using diagnostic data.

Abstract

For modeling and analyzing multi-criteria decision-making (MCDM) problems, selecting an MCDM method is an important issue. To address this issue, three methods that use interval numbers, fuzzy linguistic numbers, and belief distributions are compared based on historical decision data. Three ways are designed to flexibly compare the three methods. In the first way, criterion weights are learned from the similarity between the individual assessments and the overall assessments. In the second way, criterion weights are learned from minimizing the difference between the overall assessments and the aggregated assessments derived from the individual assessments. In the third way, three sets of criterion weights and a set of method weights are learned from minimizing the difference between the overall assessments and the aggregated results. The aggregated results are derived from unifying and combining the aggregated assessments generated using the three methods. The transformation among interval numbers, fuzzy linguistic numbers, and belief distributions is designed for their unification and a fair comparison among the three methods. The three methods are applied to help diagnose thyroid nodules for five radiologists from a tertiary hospital located in Hefei, Anhui, China. Based on the historical examination reports provided by the five radiologists, the three developed ways are used to compare the three methods. Experimental results reveal two findings. One finding is that different ways will produce different choices among the three methods. While the other is that the highest decision accuracy for the five radiologists is associated with the combination of the MCDM method and the comparison way.

Introduction

When confronting a problem, people usually need to select an appropriate one from a set of alternatives by considering conflicting criteria. Because of the complexity of the problem, it is not easy to make an effective and satisfactory tradeoff between conflicting criteria (Mohanty, Mahapatra, Mohanty, & Sthitapragyan, , 2018). Multi-criteria decision making (MCDM) methods have been developed to help people effectively consider conflicting criteria to compare the overall performance of different alternatives. In different MCDM methods, the tradeoff between multiple criteria is achieved in different ways. Various characteristics of the problem under consideration, the preferences of a decision maker, and the rationality of criterion aggregation might determine whether the tradeoff made in a specific way is satisfactory to the decision maker. This indicates that each MCDM method aims to analyze a specific class of problems in specific situations.

The rationality and applicability of a developed MCDM method are usually validated by a comparison with previous methods. This facilitates highlighting the advantages of the developed method. In existing studies, the comparison of specific MCDM methods with others is summarized in Table 1.

From Table 1, it can be concluded that the comparison between the developed method and previous methods is carried out from various perspectives. The perspectives include considering the specific preferences of a decision maker, reducing the burden on a decision maker to set parameters related to making a solution, improving the flexibility and rationality of criterion aggregation, generating a consistent and complete solution, and guaranteeing lower computational complexity. Furthermore, it can also be found that a comparison between the developed method and previous methods depends on a specific problem, two numerical examples, a qualitative analysis, or a theoretical analysis. When the focus is to validate the meaningfulness of the developed method, such a comparison might be enough. This is because different MCDM methods may be applicable to different problems, or to the same problem in different situations. However, the comparison might not be very persuasive when the performance of different methods needs to be fairly and fully compared for specific problems. Such a comparison might be more unpersuasive when the historical data concerning the individual assessments and the overall assessments under the same decision framework are available.

In some real applications, the historical data related to the individual assessments and the overall assessments are recorded. For example, in historical examination reports on the diagnosis of thyroid nodules, possible symptom description on the criteria of margin, contour, echogenicity, calcification, and vascularity (Cappelli et al., 2007, Fu et al., 2018, Kwak et al., 2011, Moon et al., 2008) and the overall diagnosis (considered as the overall assessment) have been accumulated. After the individual assessments and the overall assessments are transformed from the possible symptom description and the overall diagnosis, several MCDM methods can be used to learn the relationship between the individual assessments and the overall assessments. The learned relationship is then used to help generate diagnostic recommendations when the symptoms of a new thyroid nodule on the five criteria are observed. In this situation, the comparison like that between the developed method and previous methods in existing studies will not work. This is because the purpose of the comparison has changed. The comparison does not aim to only validate the meaningfulness of MCDM methods. Its focus is to identify appropriate methods for helping create solutions that are highly or completely consistent with the historical preferences of a decision maker. This is very meaningful in the context of historical data. A new issue emerges: selecting an appropriate method from multiple feasible methods to help create solutions on the condition that the historical data concerning the individual assessments and the overall assessments under the same decision framework are available.

To address this issue, in this paper three MCDM methods will be compared on the condition that the historical data concerning the individual assessments and the overall assessments are characterized by discrete categories. The three methods are composed of the method that uses interval numbers (Jiang et al., 2018, Lan et al., 2019, Wu et al., 2016, Zhu et al., 2017), the method that uses fuzzy linguistic numbers (Herrera and Herrera-Viedma, 2000, Herrera et al., 2009, Merigó et al., 2016, Zhou and Xu, 2016), and evidential reasoning (ER) method (Fei et al., 2019, Fei et al., 2019, Fu and Yang, 2010, Wang et al., 2006, Yang, 2001, Yang and Xu, 2002). For convenience, the former two methods are referred to as the interval method and the linguistic method. Note that there may be other methods in which their respective formats of preference information can be used to represent discrete categories. The interval method, the linguistic method, and the ER method are three representative ones, whose formats of preference information can be transformed from each other, and, thus, selected to be compared in this study. In addition to MCDM methods that can characterize discrete categories, other methods can also be compared in theory. However, this is not beneficial for a fair comparison among different methods because the formats of preference information in these methods and those in the three methods may not be unified. Further, an unfair comparison among different MCDM methods will make it difficult to select an appropriate method that can help generate solutions, which are greatly consistent with the historical preferences of a decision maker.

Three ways are developed to compare the performance of the three methods when the individual assessments and the overall assessments are transformed from the historical data. In the first way, criterion weights are learned from the similarities between the individual assessments and the overall assessments. In the second way, criterion weights are learned from minimizing the difference between the overall assessments and the aggregated assessments derived from the individual assessments. In the third way, three sets of criterion weights and a set of method weights are learned from minimizing the difference between the overall assessments and the combinational results. The results are derived from the combination of the aggregated assessments generated using the individual assessments and the three methods. The learned weights are then used to examine the performance of the three methods. The transformation among interval numbers, fuzzy linguistic numbers, and belief distributions, i.e., the format of preference information used in the ER method, is designed to ensure a fair comparison among the three methods. Based on historical examination reports collected from a tertiary hospital located in Hefei, Anhui, China, the three methods are used to help diagnose thyroid nodules, in which their performance is compared in the three ways. To the best of our knowledge, it is the first attempt to compare different MCDM methods based on large volumes of historical decision data. The developed three ways aim to flexibly compare different MCDM methods in different application contexts. A fair comparison among the three methods is guaranteed through the transformation among the three types of preference information.

The rest of this paper is organized as follows. Section 2 presents the modeling and analysis of MCDM problems using the three methods. Section 3 presents the three ways to compare the three methods based on historical data concerning individual assessments and overall assessments under the same decision framework. In Section 4, the three ways are demonstrated by their application to the comparison among the three methods based on historical examination reports on the diagnosis of thyroid nodules. The paper is concluded in Section 5.

Section snippets

Modeling and analysis of MCDM problems by using three methods

As presented in Section 1, interval numbers, fuzzy linguistic numbers, and belief distributions (BDs) are used to represent discrete categories in the three methods, respectively. By taking the diagnosis of thyroid nodules as an example, this is demonstrated. When radiologists diagnose thyroid nodules by ultrasonic examination, they use the five categories in TIRADS (thyroid imaging reporting and data system) (Horvath et al., 2017, Kwak et al., 2011, Park et al., 2009, Sahli et al., 2019),

Selection of the MCDM methods

In this section, the interval method, the linguistic method, and the ER method are compared in three ways. The three ways are developed to help flexibly select an appropriate method in different contexts. The comparison is conducted when the historical data related to the individual assessments on each criterion and the overall assessments under the same decision framework are available.

Case study

In this section, the three methods presented in Section 2 are used to help diagnose thyroid nodules and their performance is compared by using the three ways presented in Section 3. Such a comparison is not to validate the meaningfulness of the three methods, but to select an appropriate method for helping generate diagnostic recommendations of thyroid nodules that are highly consistent with the historical diagnostic preferences of radiologists. The two issues presented at the end of Section 3

Conclusions

To address uncertain MCDM problems, many methods have been developed in the literature. Almost every method is verified by real cases, numerical examples, or simulation experiments to be valid and applicable. With the help of real cases or numerical examples or both, one method is verified to have higher decision performance than other methods that can be also applicable to the cases or examples. However, the conclusions drawn from specific cases or examples may not be considered as universal

CRediT authorship contribution statement

Chao Fu: Conceptualization, Methodology, Software, Writing - original draft, Funding acquisition. Wenjun Chang: Formal analysis, Investigation, Data curation, Visualization. Weiyong Liu: Data curation, Validation, Investigation. Shanlin Yang: Resources, Supervision, Funding acquisition.

Acknowledgements

This research is supported by the National Natural Science Foundation of China (Grant Nos. 71622003, 71571060, 71690235, 71690230, and 71521001).

References (40)

  • J.C. de Borda

    A paper on elections by ballot (English translation)

  • C. Cappelli et al.

    The predictive value of ultrasound findings in the management of thyroid nodules

    QJM Monthly Journal of the Association of Physicians

    (2007)
  • S. Corrente et al.

    Handling imprecise evaluations in multiple criteria decision aiding and robust ordinal regression by n-point intervals

    Fuzzy Optimization & Decision Making

    (2017)
  • L.G. Fei et al.

    DS-VIKOR: A new multi-criteria decision-making method for supplier selection

    International Journal of Fuzzy Systems

    (2019)
  • L.G. Fei et al.

    An ELECTRE-based multiple criteria decision making method for supplier selection using Dempster-Shafer theory

    IEEE Access

    (2019)
  • C. Fu et al.

    Data-driven group decision making for diagnosis of thyroid nodule

    SCIENCE CHINA Information Sciences

    (2019)
  • C. Fu et al.

    Data-driven multiple criteria decision making for diagnosis of thyroid cancer

    Annals of Operations Research

    (2018)
  • C. Fu et al.

    A group evidential reasoning approach based on expert reliability

    European Journal of Operational Research

    (2015)
  • C. Fu et al.

    The group consensus based evidential reasoning approach for multiple attributive group decision analysis

    European Journal of Operational Research

    (2010)
  • K.X. Gong et al.

    Multiple-attribute decision making based on equivalence consistency under probabilistic linguistic dual hesitant fuzzy environment

    Engineering Applications of Artificial Intelligence

    (2019)
  • F. Herrera et al.

    A sequential selection process in group decision making with a linguistic assessment approach

    Information Sciences

    (1995)
  • F. Herrera et al.

    Computing with words in decision making: Foundations, trends and prospects

    Fuzzy Optimization and Decision Making

    (2009)
  • F. Herrera et al.

    Linguistic decision analysis: Steps for solving decision problems under linguistic information

    Fuzzy Sets and Systems

    (2000)
  • E. Horvath et al.

    An ultrasonogram reporting system for thyroid nodules stratifying cancer risk

    Journal of Clinical Endocrinology and Metabolism

    (2009)
  • E. Horvath et al.

    Prospective validation of the ultrasound based TIRADS (Thyroid Imaging Reporting And Data System) classification: Results in surgically resected thyroid nodules

    European Radiology

    (2017)
  • Y.P. Jiang et al.

    Multiple criteria decision making with interval stochastic variables: A method based on interval stochastic dominance

    European Journal of Operational Research

    (2018)
  • J.Y. Kwak et al.

    Thyroid imaging reporting and data system for us features of nodules: A step in establishing better stratification of cancer risk

    Radiology

    (2011)
  • J.B. Lan et al.

    A new linguistic aggregation operator and its application to multiple attribute decision making

    Operations Research Perspectives

    (2015)
  • J.B. Lan et al.

    Dominance degrees for intervals and their application in multiple attribute decision-making

    Fuzzy Sets and Systems

    (2019)
  • H.C. Liao et al.

    An integrated method for cognitive complex multiple experts multiple criteria decision making based on ELECTRE III with weighted Borda rule

    Omega

    (2019)
  • Cited by (10)

    • An integrated decision support system for multi-target forecasting: A case study of energy load prediction for a solar-powered residential house

      2022, Computers and Industrial Engineering
      Citation Excerpt :

      Researchers have applied multi-criteria data-driven models in a variety of topics to explore the relationship between response and independent variables. Fu et al. (2020) proposed a data-driven procedure to select MCDM methods using three techniques that utilize intervals, fuzzy linguistic numbers, and belief distributions. Çalı and Balaman (2019) utilized a fuzzy MCDM decision support system to measure customer satisfaction via online reviews and rank alternative products/services.

    • Data-driven decision-making with weights and reliabilities for diagnosis of thyroid cancer

      2022, International Journal of Machine Learning and Cybernetics
    View all citing articles on Scopus
    View full text