Decision SupportSensitivity analysis for product design selection with an implicit value function
Introduction
Existing multi-attribute decision making (MADM) or selection methods (see, for example, Olson, 1996, Yu, 1985, Lootsma, 1999) generally assume that the decision maker (DM) has in mind a value function that he/she maximizes to make the selection. (Conventionally, the term “value” is used when the attributes are deterministic, and the term “utility” when the attributes are stochastic (Keeney and Raiffa, 1976). We consider here only deterministic attributes, and use the term value in the remainder of the paper.) These methods estimate the value function by obtaining from the DM information about quantitative “preferences” which reflect the value function. Various selection methods take the preferences in various forms, e.g., relative importance of attributes (Lootsma, 1999, Saaty, 1980), comparison of design alternatives (Koksalan et al., 1984, Malakooti, 1988), or “marginal rate of substitution” between attributes (Keeney and Raiffa, 1976, Maddulapalli et al., 2005, Yu, 1985).
In general, when the DM gives the preferences, in addition to design requirements (e.g., constraints on the size, price), he/she attempts to satisfy the needs of end users or customers (e.g., a professional user of cordless power tool prefers to have more operations per battery charge, whereas a casual user prefers lower cost) (Urban and Hauser, 1993). So, if the DM does not have complete information about the end users’ needs, he/she cannot state the preferences precisely (Insua and French, 1991). The DM might also have to project into future markets. In cases with such uncertainty the DM can give only estimates of the actual preferences. Since small variations in preferences could lead to a significant change in the set of preferred designs (which might be singleton) (Korhonen et al., 1992, White, 1972), it would be useful for the DM to have an idea about the “robustness” of the set of preferred designs with respect to variation in the preference estimates (Hannan, 1981, Korhonen et al., 1992). By robustness we mean the amount of change (or variation) allowed between the actual preferences and the preference estimates before the set of preferred designs is affected (i.e., gains additional member(s)).
Finding the degree of robustness of the preferred designs to preference variation is generally referred to as sensitivity analysis in the literature (Insua and French, 1991). One advantage of sensitivity analysis is that if the DM is confident that the set of preferred designs is sufficiently robust, then he/she can select from that set. Alternatively, using sensitivity analysis, the DM can get an idea of what other designs might be preferred if the preference variation increases.
Existing literature in sensitivity analysis addresses cases where the DM’s value function is presumed to be explicitly known (e.g., known polynomial function of attributes with unknown parameters (Keeney and Raiffa, 1976, Lootsma, 1996). Sage (1981) studied and formalized the allowed errors in the estimation and elicitation of probabilities and utilities before which the preferred design is affected. Barron and Schmidt (1987) proposed two procedures: entropy-based and least square (i.e., L2-metric) to calculate the minimum variation required between the actual weights and the estimates of weights to change the most preferred design when the value function is linear. Ringuest (1997) later extended the L2-metric of Barron and Schmidt (1987) to an LP-metric. Mareschal (1988) proposed an approach for finding the “weight stability interval”, which consists of all possible weights that maintain the rank order obtained using the original estimates of weights.
Insua and French (1991) proposed some distance based tools to identify the possible competitors to the current most preferred design when the DM’s preferences change. Antunes and Climaco (1992) proposed a sensitivity analysis approach for their TRIMAP method. However, this approach is applicable only when the number of attributes is three or less, which is a significant limitation. Triantaphyllou and Sanchez (1997) proposed a sensitivity analysis approach and applied it to popular MADM methods like weighted sum model, weighted product model, and analytical hierarchy process (Saaty, 1980). Ma et al. (2001) presented a method for finding the “weight-set” that contains all possible ranges of weights of an additive value function when the rank order of alternatives is given. Triantaphyllou and Shu (2001) studied the number of feasible rankings that are possible, assuming an additive value function, for the given set of design alternatives, when the weights of the criteria are allowed to change.
Although MADM literature describes significant research on sensitivity analysis when the value function is presumed, it is well known that presuming a form for the value function is restrictive and applicable only to special cases (Maddulapalli et al., 2005, Malakooti, 1988, Thurston, 2001). In this paper we present a broadly applicable concept for calculating a robustness index for a set of preferred designs. The concept can be used with any iterative selection scheme that chooses a trial design for each iteration, and uses the DM’s estimates of preference parameters at that trial design to eliminate some design options which have lower value than the trial design (Maddulapalli et al., 2005, Malakooti, 1988). Such schemes are, in general, applicable to cases where the DM’s value function is implicit. The output of such schemes is called the set of non-eliminated trial designs.
Our robustness index is a metric of the allowed preference variation for which the set of non-eliminated trial designs will not change (i.e., all eliminated designs remain eliminated). The DM, through experience, can decide if the robustness index of the set is high enough to make a final selection from the present set, or can improve the precision of the preferences, or otherwise cope with the imprecision.
We present our sensitivity analysis concept and an implementation of it using our previously published (Maddulapalli et al., 2005) iterative method for design selection with an implicit value function that need be only differentiable, quasi-concave and non-decreasing with respect to the attributes. Our concept can also be used with iterative methods that use DM’s preferences to estimate the unknown parameters of the presumed value function (e.g., Cobb–Douglas function (Takayama, 1993)).
The organization of this paper is as follows. In Section 2 we provide some definitions for the main terminology used in the paper. In Section 3 we present an overview of our concept for finding a robustness index. Section 4 describes the implementation of the concept using our design selection method (Maddulapalli et al., 2005). Next, in Section 5 we discuss our algorithm for finding the robustness index. In Section 6, we demonstrate the application of our method with the help of two examples. Finally we conclude with a summary in Section 7.
Section snippets
Definitions
In this section we give definitions and some pertinent properties for the important terms used throughout the paper.
Overview of concept for finding robustness index
Our concept for sensitivity analysis is applicable to iterative selection methods, (e.g., Maddulapalli et al., 2005, Malakooti, 1988), which choose a trial design DT at each iteration, and examine every other design D+ in the input set to eliminate those having lower value than DT. The output of such a method is a set of non-eliminated trial designs DNTD. Fig. 3 shows the flowchart of our concept for calculating three successive metrics, culminating in the “robustness index” of DNTD.
For each D+
Implementation of concept for finding robustness index
In this section, we describe the implementation of the concept (recall Fig. 3) in our previously published (Maddulapalli et al., 2005) method for design selection with an implicit value function that need be only differentiable, quasi-concave and non-decreasing with respect to the attributes. Because the DM’s value function is assumed non-decreasing with respect to attributes, for selection, it is enough to consider only those designs that are Pareto optimal from the original set of design
Algorithm for finding robustness index
Our algorithm for finding the robustness index has the following steps.
- Step 1:
Set the iteration number to one (i.e., i = 1) and pick a starting trial design, DT1, from the set of design alternatives. We choose DT1 either as an alternative that would have maximum value if the value function were linear with equal importance to the attributes, or as a random pick.
- Step 2:
Query the DM for the preference (MRS) estimates at the current trial design DTi and find the gradient of the value function at DTi (recall
Examples
As a demonstration, we tested our robustness index concept by applying our algorithm to two examples: an engineering example and a numerical example. The engineering example is a two-attribute problem and involves the selection of a payload design for an undersea autonomous vehicle. The second example is a three-attribute problem and involves selection from fifty randomly generated designs. The engineering example graphically demonstrates the working of our algorithm. The numerical example
Summary
In this paper, we presented a concept for sensitivity analysis in product design selection when the DM gives only estimates of the actual preferences. Our concept is applicable to the class of iterative selection methods which eliminate some design options at the trial design chosen for the current iteration. Such methods are generally used when the DM’s value function is implicit rather than known. In our concept, we calculate three successive metrics, culminating in the “robustness index” for
Acknowledgements
The work presented in this paper was supported in part by a contract from Indian Head Division, Naval Surface Warfare Center. Indian Head Division was funded by the Office of Naval Research. Such support does not constitute an endorsement by the funding agency of the opinions expressed in the paper.
References (29)
- et al.
Sensitivity analysis in MCDM using the weight space
Operations Research Letters
(1992) On the decomposition of value functions
Operations Research Letters
(1998)- et al.
A framework for sensitivity analysis in discrete multi-objective decision-making
European Journal of Operational Research
(1991) - et al.
Multiple criteria decision support—a review
European Journal of Operational Research
(1992) A model for the relative importance of criteria in multiplicative AHP and SMART
European Journal of Operational Research
(1996)Weight stability intervals in multicriteria decision aid
European Journal of Operations Research
(1988)LP-metric sensitivity analysis for single and multi-attribute decision analysis
European Journal of Operations Research
(1997)Sensitivity analysis in systems for planning and decision support
Journal of Franklin Institute
(1981)- et al.
On the maximum number of feasible ranking sequences in multi-criteria decision making problems
European Journal of Operational Research
(2001) - et al.
Sensitivity analysis of additive multi-attribute value models
Operations Research
(1987)
Nonlinear Programming: Theory and Algorithms
Quality assisted multiobjective multidisciplinary genetic algorithms
AIAA Journal
Obtaining nondominated priority vectors for multiple objective decision making problems with different combinations of cardinal and ordinal information
IEEE Transaction on Systems, Man, and Cybernetics
Decision with Multiple Objectives Preferences and Value Tradeoffs
Cited by (16)
Knowledge evaluation in product lifecycle design and support
2014, Knowledge-Based SystemsCitation Excerpt :In product design process, designers are assumed to have in mind a utility function that he/she maximizes to make the selection in multi-attribute decision making. For more complex cases, Maddulapalli et al. [22] have presented an algorithm for calculating the “robustness index” in case where decision makers give estimates of the end users’ needs preferences. The algorithm is applicable to cases where product designers do not have enough information about the precise end users’ preferences or decision makers’ value function is implicit.
An integrated quality-function-deployment and stochastic-dominance-based decision-making approach for prioritizing product concept alternatives
2022, Complex and Intelligent SystemsResponse reliability and risk analysis of users under time-of-use price
2022, IET Generation, Transmission and DistributionAn integrated decision-making approach for designing and selecting product concepts based on QFD and cumulative prospect theory
2018, International Journal of Production Research