A framework of cloud service selection with criteria interactions

https://doi.org/10.1016/j.future.2018.12.005Get rights and content

Highlights

  • A novel MCDM framework for cloud service selection.

  • Service selection based on criteria priority orders and interaction types.

  • An executable approach to build knowledge-based ground truth to validate MCDMs.

  • Identify disadvantages and limitations of linear MCDMs and manual service ranking.

  • Substantial experiments based on real datasets to validate the proposed method.

Abstract

Existing cloud service selection techniques assume that service evaluation criteria are independent. In reality, there are different types of interactions between criteria. These interactions influence the performance of a service selection system in different ways. In addition, a lack of measurement indices to validate the performance of service selection methods has hindered the development of decision making techniques in the service selection area. This paper addresses these critical issues of modeling the interactions between cloud service selection criteria, and designing indices to validate service selection methods. In this paper, we propose a Cloud Service Selection with Criteria Interactions framework (CSSCI) that applies a fuzzy measure and Choquet integral to measure and aggregate non-linear relations between criteria. We employ a non-linear constraint optimization model to estimate the Shapley importance and criteria interaction indices. In addition, we design a priority-based CSSCI (PCSSCI) to solve service selection problems in the situation where there is a lack of historical information to determine criteria relations and weights. Furthermore, we discuss an approximate solution for CSSCI to reduce its computing complexity. Finally, we design three indices to validate the cloud service selection methods. The experimental results preliminarily prove the technical advantage of the proposed models in contrast to several existing models.

Introduction

With the development of cloud computing and the proliferation of cloud services on the Internet, cloud service selection has become an area undergoing intense study [1]. Existing service selection techniques assume selection criteria are independent [2], [3]. This assumption does not take into account the fact that there are different types of non-linear relations between criteria. In reality, components in a complex system interact with each other in different forms. Those different types of interactions influence the performance of the whole system [4]. This study builds a non-linear modeling mechanism to solve cloud service selection where the selection criteria may have different types of interactions.

There are three types of relations between criteria: positively interacting (or supporting), negatively interacting (or conflicting) and independent [5]. In this paper, we mainly explore how criteria interrelationships (we call them interactions) influence service selection results. If two criteria positively interact with each other, they are similar to each other from a specific perspective (e.g. function), so they influence each other’s utilities positively. However, their interactions influence the overall utility of a service negatively. Here we use utilities to measure the criterion performance or the satisfaction degrees of service users. If two criteria conflict with each other, the increase of a criterion utility causes the decrease of the utility of the other criterion. If two criteria do not have any relations, they are independent of each other.

We use a real scenario to explain the significance of modeling criteria interactions in service selection. Assume a user Jame wants to select a SaaS from two services with similar functions. He uses three criteria to evaluate services: cost (ct), availability (av) and successability(su). His preferences are: (1) both av and su are 2 times more important than ct. However (2) if both services have high av, and not very low su, he will prefer the service which has lower ct. Based on the user’s preference (1), we assign weights for criteria: wct=0.2, wav=0.4 and wsu=0.4. Assume the monthly payments and QoS performance of three services are: service1 s1(ct,av,su)=($8,99%,95%) and service2 s2(ct,av,su)=($6,99%,80%). We standardize them using this procedure: for the type of benefit criteria (i.e. the higher value of a benefit criteria, the better its performance, e.g. av and su), the standardized value of av is: sds1(av)=s1(av)max(s1(av),s2(av),s3(av)); and for the type of cost criteria (i.e. the higher value of a cost criteria, the worse its performance, e.g. ct), the standardized value is: sds1(ct)=1s1(av)max(s1(ct),s2(ct),s3(ct)). The standardized values of the three services are: sds1(ct,av,su)=(0,1,1) and sds2=(0.25,1,0.81). The simple weighted addition (SWA) [6] will recommend service1 based on their simple aggregated scores: u(s1)=0.20+0.41+0.41=0.8>u(s2)=0.20.25+0.41+0.40.81=0.77. However, the user’s preference (2) indicates that he might prefer service2. This inaccurate recommendation of SWA is because of its inability to aggregate the non-linear relations between preferences (1) and (2), which causes a redundant consideration of the high performance of av and su of service2, and in turn increases the evaluation score of service2. On the other hand, according to preference (2), av and su are both important to the user (he does not want very low su), so it is necessary to consider both of them in a service selection. Therefore, it is important to build a non-linear model to evaluate services based on the complexity of users’ preferences and criteria interactions.

In this paper, we propose a cloud service selection framework that models criteria interactions for service selection. We name this framework Cloud Service Selection with Criteria Interactions (CSSCI). We apply fuzzy measure (FM) [7] and 2-order additive Choquet integral [8] to estimate the overall utility of a service based on the importance of single criteria and the influence of two interactive criteria on the final decision. We use a non-linear constraint optimization model to estimate the importance of single criteria and the interaction indices between criteria. To solve the optimization model, we use a pair-wise comparison method to determine the criterion importance, and define an interaction ratio to assist in estimating the interaction indices. As there may be situations where insufficient historical information exists to support building criteria interactions, we design a priority-based CSSCI (PCSSCI), in which users only need to provide a priority order of criteria and subjectively define types of criteria interactions. PCSSCI uses an interactive interpretive structural modeling (I-ISM) to interactively establish consistent criteria interactions.

To validate the proposed methods, we design three algorithms to evaluate the efficiency of CSSCI, which are: average ratio of a service ranking similar to the service ranking of experts (ar), Kendall Tau distance ratio (ktdr) among rankings of a set of multi-criteria decision making (MCDM) methods, and stability of service ranking (st). The experiment results show that the ar of CSSCI is around 50%, 50% and 10% better than that of SWA [6], VlseKriterijuska Optimizacija I Komoromisno Resenje (VIKOR) [9] and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) [10] respectively; the ktdr of CSSCI is around 2% better than the ktdrs of SWA and TOPSIS, and 3.5% better than the ktdr of VIKOR; and the st of CSSCI performs better than SWA and TOPSIS by around 10%. In particular, SWA, TOPSIS and VIKOR are not capable of modeling non-linear relations between criteria based on users’ interactive preferences. As it is time-consuming for CSSCI to precisely solve an optimization model with multiple constraint conditions, we propose an approximate solution of CSSCI (apprCSSCI), which uses the initial weights of single criteria and tunes a parameter for determining the interaction indices. We compare the service ranking results of apprCSSCI with those of CSSCI. The comparison results show that the approximate solution has a very low Kendall Tau distance (ktd) to the precise solution when the parameter of apprCSSCI is set appropriately. The distinctive contributions of this paper are:

  • We propose a novel MCDM framework for cloud service selection, which models users’ nonlinear preferences based on criteria interactions.

  • We deal with service selection in situations where there is a lack of historical information, and introduce an objective service selection method that only requires users to provide criteria priority orders and interaction types.

  • We design three algorithms to rigorously evaluate the proposed framework. In particular, we propose the first simple and executable approach to establish an expert-knowledge-based ground truth to validate MCDM methods, which is a significant contribution as a lack of expert-knowledge-based labels have hindered the usage of MCDMs in the service selection literature.

  • We identify the disadvantages of classical linear MCDM methods for solving service selection problems, and identify the limitations of manual service ranking with respect to (w.r.t.) high criteria numbers.

  • We conduct substantial experiments to validate the proposed methods based on the real datasets of SaaSs and cloud computing services.

These contributions have not been addressed in the existing literature.

This paper is organized as follows: Section 2 introduces related work; Section 3 introduces the preliminary knowledge; Section 4 defines the problem of cloud service selection with criteria interactions, and overviews the CSSCI; Section 5 describes the CSSCI; Section 6 introduces the PCSSCI; Section 7 presents the experiment processes and results, and discusses the approximate CSSCI; and Section 8 concludes this paper.

Section snippets

Related work

In this section, we review the state-of-the-art of Web and cloud service selection approaches over the past five years. We categorize these works into two groups: MCDM-based and non-MCDM-based service selection. After introducing the literature, we analyze the differences between our work and the existing works.

MCDM and MAUT methods

Assume an MCDM problem is to select an alternative (or solution) from m alternatives: a1,,am, based on values (or performance, utilities ) of n criteria (or attributes, goals): c1,,cn, this decision-making problem can be modeled as a m×n decision matrix DM=, where xij,i[1,m],j[1,n] is the value of criterion cj of alternative ai [44].

Simple weighted addition (SWA) [45] is the most simple and popular MAUT method. If each criterion has a weight, e.g. wj, j[1,n], the overall SWA value of

Problem definition and framework overview

In this section, we define the cloud service selection problem in CSSCI and overview the framework of CSSCI.

Cloud service selection with criteria interactions

In this section, we present the steps of CSSCI in detail in a methodological manner and describe a running example of using CSSCI.

Priority-based CSSCI

Section 5 introduces service rankings based on CSSCI in cases where decision makers have enough datasets to determine the interaction indices of criteria. However, in some cases, we do not have enough historical data, so we have to select services based on experts’ or users’ preferences. In this section, we propose a method for service selection in cases where there is a lack of objective coefficients between criteria. This method requires users to provide priority orders of criteria and the

Experiments and analysis

We conduct experiments in Matlab and R on a 64-bit Windows 10 platform with an Intel Core i7-6700HQ CPU and 16 GB RAM. We use both real [56] and simulated criteria datasets of SaaSs for CSSCI validation, which include eight criteria ( av,th,su,re,la,res,rscs, and ct in Table 1) to evaluate 2507 services. Section 5 gives a detailed description of this dataset.

In addition, we use examples to analyze the performance of priority-based CSSCI based on a QoS dataset of cloud computing services. We

Conclusion

In this paper, we proposed a CSSCI framework to evaluate the influence of different types of criteria interactions on cloud service selection results. We applied the FM, CI and non-linear constraint optimization techniques to identify the Shapley importance and interaction indices of criteria. In addition, we proposed a priority-based CSSCI to deal with the case where there is a lack of historical information to determine the weights of single criteria and interaction indices. In the

Funding

This work is supported by the National Natural Science Foundation of China (Grants No 61702274) and the Natural Science Foundation of Jiangsu Province (Grants No BK20170958), PAPD and partially supported by the China-USA Computer Science Researcher Center .

Le Sun is a Lecturer at the School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China. She received her Ph.D. degree from Victoria University, Australia, in 2016. Her research interests include service-oriented computing and cloud computing.

References (69)

  • WangS. et al.

    Efficient and reliable service selection for heterogeneous distributed software systems

    Future Gener. Comput. Syst.

    (2017)
  • SomuN. et al.

    A computational model for ranking cloud service providers using hypergraph based techniques

    Future Gener. Comput. Syst.

    (2017)
  • DingS. et al.

    Combining qos prediction and customer satisfaction estimation to solve cloud service trustworthiness evaluation problems

    Knowl.-Based Syst.

    (2014)
  • DingS. et al.

    Utilizing customer satisfaction in ranking prediction for personalized cloud service selection

    Decis. Support Syst.

    (2017)
  • DingS. et al.

    Time-aware cloud service recommendation using similarity-enhanced collaborative filtering and arima model

    Decis. Support Syst.

    (2018)
  • XuX. et al.

    S-abc: a paradigm of service domain-oriented artificial bee colony algorithms for service selection and composition

    Future Gener. Comput. Syst.

    (2017)
  • OpricovicS. et al.

    Compromise solution by mcdm methods: a comparative analysis of vikor and topsis

    European J. Oper. Res.

    (2004)
  • MathiyazhaganK. et al.

    An ism approach for the barrier analysis in implementing green supply chain management

    J. Cleaner Prod.

    (2013)
  • TorraV. et al.

    The interpretation of fuzzy integrals and their application to fuzzy systems

    Internat. J. Approx. Reason.

    (2006)
  • GrabischM.

    K-order additive discrete fuzzy measures and their representation

    Fuzzy Sets and Systems

    (1997)
  • CombarroE.F. et al.

    On the structure of the k-additive fuzzy measures

    Fuzzy Sets and Systems

    (2010)
  • YagerR.R. et al.

    Fuzzy measures in multi-criteria decision making

    Procedia Comput. Sci.

    (2015)
  • KoczkodajW.

    A new definition of consistency of pairwise comparisons

    Math. Comput. Modelling

    (1993)
  • RezaeiJ.

    Best-worst multi-criteria decision-making method

    Omega

    (2015)
  • LévayP.Z. et al.

    The effect of fiscal incentives on market penetration of electric vehicles: a pairwise comparison of total cost of ownership

    Energy Policy

    (2017)
  • LiuS. et al.

    Decision making for the selection of cloud vendor: an improved approach under group decision-making with integrated weights and objective/subjective attributes

    Expert Syst. Appl.

    (2016)
  • ZhangN. et al.

    Extension of vikor method for decision making problem based on hesitant fuzzy set

    Appl. Math. Model.

    (2013)
  • WangZ.-X. et al.

    Evaluation of the provincial competitiveness of the chinese high-tech industry using an improved topsis method

    Expert Syst. Appl.

    (2014)
  • ChenY. et al.

    Spatial sensitivity analysis of multi-criteria weights in gis-based land suitability evaluation

    Environ. Modell. Softw.

    (2010)
  • L SaatyT.

    The analytic network process

    Iran. J. Oper. Res.

    (2008)
  • MarichalJ.L.

    An axiomatic approach of the discrete choquet integral as a tool to aggregate interacting criteria

    IEEE Trans. Fuzzy Syst.

    (2000)
  • MengF. et al.

    Entropy and similarity measure of atanassovs intuitionistic fuzzy sets and their application to pattern recognition based on fuzzy measures

    Pattern Anal. Appl.

    (2016)
  • WuJ. et al.

    2-additive capacity identification methods from multicriteria correlation preference information

    IEEE Trans. Fuzzy Syst.

    (2015)
  • AkbarizadeZ. et al.

    Ranking cloud service providers using swara and vikor (a case of irancell company)

    Int. J. Inf. Secur. Syst. Manag.

    (2017)
  • Cited by (55)

    • A decision framework with nonlinear preferences and unknown weight information for cloud vendor selection

      2023, Expert Systems with Applications
      Citation Excerpt :

      Al-Faifi et al. (Al-Faifi, Song, Hassan, Alamri, & Gumaei, 2019) performed CVS with a hybrid technique encompassing clustering and DEMATEL-ANP with smart data. Sun et al. (Sun, Dong, Khadeer, Khadeer, & Liu, 2019) presented a Choquet integral-nonlinear programming integrated approach for CVS with proper attribute interaction. Lai et al. (Lai et al., 2020) introduced Z number-based double normalized aggregation with Gini index for weight assessment under sustainable CVS context.

    • A Fuzzy-Multi Attribute Decision Making approach for efficient service selection in cloud environments

      2022, Expert Systems with Applications
      Citation Excerpt :

      Cloud computing, the future Internet computing paradigm has brought a technological transformation to various business models through Anything-as-a-Service (XaaS), thereby delivering large-scale computational resources in a ‘Pay-As-You-Use’ manner (Loukis et al., 2019). With the extreme benefits like disaster recovery, rapid-elasticity, cost-efficiency, etc., the cloud services are available at “configuration-costs” packages from a wide range of Cloud Service Providers (CSPs) offering a similar set of cloud services (Hussain et al., 2020; Raghavan & Chandrasekaran, 2021; Sun et al., 2019). The presence of diverse user-QoS requirements along with the numerous functionally similar cloud services in the cloud ecosystem has resulted in a unique and interesting research problem known as user requirement specific cloud service selection.

    • An integrated decision model for cloud vendor selection using probabilistic linguistic information and unknown weights

      2022, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      Azadi et al. (2019) developed a slack-based data envelopment approach for CV selection with a variable and constant return to scale assumptions. Sun et al. (2019) prepared a model with criteria interactions for CV selection by adopting fuzzy measures and Choquet integral. Hussain et al. (2020a) adopted an integrated approach with the best-worst method and Copeland’s measure for apt selection of CVs for an e-commerce company.

    View all citing articles on Scopus

    Le Sun is a Lecturer at the School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China. She received her Ph.D. degree from Victoria University, Australia, in 2016. Her research interests include service-oriented computing and cloud computing.

    Hai Dong is a Lecturer and Vice-Chancellor’s Research Fellow at the School of Science in RMIT University, Melbourne, Australia. He received a Ph.D. and a Master’s degree from Curtin University of Technology, Australia, in 2010 and 2006. His research interests include service-oriented computing, semantic search, ontology, and cloud computing. He has published over 50 research publications in leading international journals and conferences.

    Omar Khadeer Hussain is currently a Senior Lecturer at the School of Business, UNSW Canberra. Prior to joining the School in February 2014 as a Lecturer, he worked as a Senior Research Fellow at Curtin University. Omar’s research areas of interests are Logistics and Supply Chain Management, Distributed and Grid Systems, Decision Support, Group Support Systems and their applications to Logistics areas. Omar’s research has been published in international journals such as The Computer Journal, Journal of Intelligent Manufacturing etc. He has won university and faculty level awards from his research and as the main supervisor has supervised 7 Ph.D. students to completion. In 2011, he was awarded with an APDI Fellowship from the ARC on a Linkage project with Prof Elizabeth Chang as the lead CI.

    Farookh Khadeer Hussain is an Associate Professor at the School of Software, University of Technology Sydney. He is an Associate Member of the Advanced Analytics Institute and a Core Member of the Centre for Quantum Computation and Intelligent Systems. He is a member of the Institute of Electrical and Electronics Engineers.

    Alex X. Liu is a Professor in Department of Computer Science and Engineering, Michigan State University. He received his Ph.D. degree in Computer Science from The University of Texas at Austin in 2006. He received the IEEE & IFIP William C. Carter Award in 2004, the National Science Foundation CAREER Award in 2009, and the Michigan State University Withrow Distinguished Scholar Award in 2011. His research focuses on networking and security.

    View full text