Spatial sensitivity analysis of multi-criteria weights in GIS-based land suitability evaluation

Abstract

With growing interest in extending GIS to support multi-criteria decision-making (MCDM) methods, enhancing GIS-based MCDM with sensitivity analysis (SA) procedures is crucial to understand the model behavior and its limitations. This paper presents a novel approach of examining multi-criteria weight sensitivity of a GIS-based MCDM model. It explores the dependency of model output on the weights of input parameters, identifying criteria that are especially sensitive to weight changes and to show the impacts of changing criteria weights on the model outcomes in spatial dimension. A methodology was developed to perform simulations where the weights associated with all criteria used for suitability modelling were varied one-at-a-time (OAT) to investigate their relative impacts on the final evaluation results. A tool which incorporates the OAT method with the Analytical Hierarchy Process (AHP) within the ArcGIS environment was implemented. It permits a range of user defined simulations to be performed to quantitatively evaluate model dynamic changes, measures the stability of results with respect to the variation of different parameter weights, and displays spatial change dynamics. A case study of irrigated cropland suitability assessment addressing the application of the new GIS-based AHP-SA tool is described. It demonstrates that the tool is spatial, simple and flexible.

Introduction

Multi-criteria decision making (MCDM) is primarily concerned with how to combine the information from several criteria to form a single index of evaluation. GIS are best suited for handling a wide range of criteria data at multi-spatial, multi-temporal and multi-scale from different sources for a time-efficient and cost-effective analysis. Therefore, there is growing interest in incorporating GIS capability with MCDM processes. Spatial MCDM has also become one of the most useful methods for landuse and environmental planning, as well as water and agricultural management (Davidson et al., 1994, Ahamed et al., 2000, Joerin et al., 2001, Ceballos-Silva and López-Blanco, 2003, Sicat et al., 2005, Chen et al., 2007). As a result, the request for GIS models and tools supporting collaborative decisions has increased over the last decade (Kollias and Kalivas, 1998, Karnatak et al., 2007, Reshmidevi et al., 2009, Chen et al., 2009). GIS-based MCDM involves a set of geographically defined basic units (e.g. polygons in vectors, or cells in rasters), and a set of evaluation criteria represented as map layers or attributes. Based on a particular ranking schema, it ultimately informs a spatially complex decision process by deriving a utility of these spatial entities through overlaying the criterion maps according to the attribute values and decision maker’s preferences using a set of weights. Therefore, besides criteria selection, criteria weights severely impact the results of the MCDM.

Using Analytical Hierarchy Process (AHP) is one of the most popular methods to obtain criteria weights in MCDM (Saaty, 1977, Saaty, 1980, Saaty and Vargas, 1991, Wu, 1998, Ohta et al., 2007). The AHP has been employed in the GIS-based MCDM (Carver, 1991, Malczewski, 1999a, Malczewski, 1999b, Malczewski, 2004, Makropoulos et al., 2003, Marinoni, 2004, Marinoni et al., 2009). It calculates the needed weights associated with criterion map layers by the help of a preference matrix where all identified relevant criteria are compared against each other with preference factors. Then the weights can be aggregated with the criterion maps in a way similar to weighted combination methods. GIS-based AHP is popular because of its capacity to integrate a large amount of heterogeneous data and the ease in obtaining the weights of a large number of criteria, and therefore, it has been applied in tackling a wide variety of decision making problems (Tiwari et al., 1999, Nekhay et al., 2008, Hossain and Das, 2009).

It should be recognised that MCDM-derived rankings are often conditional. The uncertainty can come from many different sources, such as original data, data processing, criteria selection and their thresholds. Criteria weights are often the greatest contributor to controversy and uncertainty. This could be because decision makers are not absolutely aware of their preferences regarding the criteria, and may be because nature and scale of the criteria is not known. Or, especially when multiple decision makers are involved, it is often not possible to derive only one set of weights, but ranges of weights, and thus more than one set of results.

Sensitivity analysis (SA) explores the relationships between the output and the inputs of a modelling application. It is “the study of how the variation in the output of a model (numerical or otherwise) can be apportioned, qualitatively or quantitatively, to different sources of variation, and how the model depends upon the information fed into it” (Saltelli et al., 2000). SA is crucial to the validation and calibration of numerical models. It can be used to check the robustness of the final outcome against slight changes in the input data (Ticehurst et al., 2003, Newham et al., 2003, Merritt et al., 2005, Zoras et al., 2007). There are some well established techniques for SA, ranging from differential to well-known Monte Carlo analysis, from measures of importance to sensitivity indices, and from regression or correlation methods to variance-based techniques (Bootlink et al., 1998, Hyde et al., 2004, Manache and Melching, 2008). A thorough review of many SA methods can be found in Saltelli et al. (2000) and Campolongo et al. (2000).

The SA procedures can help reduce uncertainty in MCDM and the stability of its outputs by illustrating the impact of introducing small changes to specific input parameters on evaluation outcomes (Archer et al., 1997, Crosetto et al., 2000, Crosetto and Tarantola, 2001, Ravalico et al., 2010). A variety of different procedures, most taking non-spatial forms, exist for dealing with the SA issues in MCDM. Several examples are described in the literature. Hill et al. (2005) analysed the ASSESS AHP and the role of quantitative methods in spatial decision support. Hyde and Maier (2006) developed a spreadsheet program that examines the robustness of a model run results obtained using MCDM. It is perhaps more common to use SA for the analysis of changes in the weights given to the criteria rather than checking for changes regarding the criteria values. For example, Janssen (1996) investigated sensitivity to changes in the importance of criteria within the decision rule. Hyde et al. (2005) proposed a sensitivity analysis to analyse the effects of uncertainties associated with the criteria weights in multi-criteria decision analysis for water resource decision making. However, SA is not a common practice in the field of spatial MCDM. It is still largely absent or rudimentary for MCDM studies. Delgado and Sendra (2004) conducted a review on how SA has been applied to GIS-based MCDM models. It indicated little attention had been paid to the evaluation of the final results from these model simulations. In addition, the SA method most frequently used is based on the variation of the weights of criteria implied in the process to test whether it significantly modifies the results obtained. Perhaps the most critical shortcoming of SA procedures found in limited GIS-MCDM applications is the lack of insight they provide into the spatial aspects of weight sensitivity. It is recommended that SA procedures should permit weight sensitivity to be visualized geographically and to facilitate the spatial analysis of sensitivity, where appropriate (Feick and Hall, 2004).

This paper addresses these documented shortcomings by presenting a new approach for investigating the spatial dimension of multi-criteria weight sensitivity. It implements a generic SA methodology in a GIS-based AHP-MCDM model, which serves as an AHP-SA tool to be used for examining the sensitivity of MCDM evaluations to criteria weight changes, and subsequently visualizing the spatial change dynamics relative to decision making problems. The approach is demonstrated using GIS-based multi-criteria land suitability assessment for potential irrigated agriculture in the Macintyre Brook catchment of Queensland, Australia.