‘Nearly’ universally optimal designs for models with correlated observations
Introduction
Consider the common linear regression model where are linearly independent continuous functions, denotes a random error process or field, are unknown parameters, and is the explanatory variable, which varies in a compact design space . We assume that observations, say , can be taken at experimental conditions to estimate the parameters in the linear regression model (1). Suppose that is a stochastic process with Throughout this paper, we call the function a covariance kernel. An important case appears when the error process is stationary and the covariance kernel is of the form . If , the function is called the correlation function, and, if as , the function is a singular covariance function. Regression models with correlated errors are often used in practice, for example, in analysis of spatial models (Fedorov, 1996, Müller, 2007), computer experiments (Bates et al., 1996), and nonlinear models of chemical processes (Dette et al., 2010, Ucinski and Atkinson, 2004).
If observations, say , are available at experimental conditions , and the covariance kernel is known, then the vector of parameters can be estimated by the weighted least squares method, that is, by , where is an matrix and is an matrix. We assume that points are such that matrices and are invertible. Note that the estimator is the best unbiased linear estimator (BLUE) of , and its variance–covariance matrix is given by If the correlation structure of the process is not known, one usually uses the ordinary least squares estimator , which has the covariance matrix
An exact experimental design is a collection of points from the design space , which defines the time points or experimental conditions where observations are taken. Optimal designs for weighted or ordinary least squares estimation minimize a functional of the covariance matrix of the weighted or ordinary least squares estimator, respectively, and numerous optimality criteria have been proposed in the literature to discriminate between competing designs (see Pukelsheim, 2006).
Exact optimal designs for specific linear models with correlated observations have been investigated in Dette et al. (2008b), Kiseľák and Stehlík (2008), and Harman and Štulajter (2010). Because even in simple models exact optimal designs are difficult to find, most authors use asymptotic arguments to determine efficient designs for the estimation of the model parameters (see Sacks and Ylvisaker, 1966, Sacks and Ylvisaker, 1968, Bickel and Herzberg, 1979, or Zhigljavsky et al., 2010).
Sacks and Ylvisaker, 1966, Sacks and Ylvisaker, 1968 and Näther (1985, Chapter 4) assumed that the design points are generated by the quantiles of a distribution function; that is, , where the function is the inverse of a distribution function. Let denote a normalized design supported at points with the weight assigned to each point. Then the covariance matrix of the least squares estimator given in (3) can be represented as where the matrices and are defined by respectively (the integration is always taken over the set ), and denotes the vector of regression functions. We call any probability measure on an approximate design or simply a design; however, its interpretation in practice is different from the one given in Kiefer (1974), where, in the case of a discrete design , the weight means the relative proportion of observations performed at the point . In the case of correlated errors, only one realization of a stochastic process is usually observed, implying that no replication of design points is needed, and in practice design points are computed as quantiles of the cumulative distribution function defined by ; this rule applies independently of whether is a discrete or continuous probability measure (or a mixture of the two). If some points in the collection of quantiles replicate (this can happen if is small and the optimal design is discrete), we can replace the points which replicate by other points that are near to them. The definitions of the matrices and can be extended to an arbitrary design , provided that the corresponding integrals exist. The matrix is called the covariance matrix for the design , and can be defined for any probability measure supported on the design space such that the matrices and are well defined. This set will be denoted by . We assume that the design set has enough points so that the set is non-empty; that is, there exists at least one design .
Optimal designs for regression models with dependent data have been investigated mainly for the location scale model. The difficulties in a general development of the optimal design theory for correlated observations can be explained by the different structure of the covariance of the least squares estimator in model (1), which is of the form . As a consequence, the corresponding design problems are in general not convex (except for the location scale model where and ). Recently, Dette et al. (2011) derived universally optimal designs for regression models of arbitrary dimension if the corresponding regression functions are eigenfunctions of an integral operator defined by the covariance kernel of the error process. For example, the design with arcsine density is universally optimal for the polynomial model with logarithmic covariance kernel. On the other hand, there are many situations where this assumption is not satisfied, and in these cases there may not exist a universally optimal design.
The present paper is devoted to the numerical construction of ‘nearly’ universally optimal designs for regression models in such situations. This means that we consider model (1) with parameters in the case where a universally optimal design does not exist. In Section 2, we introduce a new optimality criterion which reflects the distance between a given design and an ideal universally optimal design. A necessary condition for the optimality of a given design is established in Section 3, and an algorithm for its numerical determination is proposed in Section 4. Finally, some illustrative examples are given in Section 5, where we calculate ‘nearly’ universally optimal designs for a quadratic regression model and a nonlinear model with various correlation functions. The results indicate that the new ‘nearly’ universally optimal designs have good efficiencies with respect to common optimality criteria.
Section snippets
A new optimality criterion
Throughout this paper we assume that the kernel in (6) is continuous at all points except possibly the diagonal points . We also assume that for at least one pair with . Singular kernels appear naturally if the approach in Bickel and Herzberg (1979) for the approximation of the covariance matrix in (3) is extended such that the variance of the observations also depends on the sample size (see Zhigljavsky et al. (2010) for details and Dette et al. (2011)
Necessary condition for optimality
In the case of correlated observations, optimality criteria are generally not convex; see Zhigljavsky et al. (2010). Therefore, standard optimality theorems do not give full characterizations of optimal designs. These results only provide necessary conditions for the optimality of a design. Theorem 1 below is not an exception, and it only gives a necessary condition for -optimality.
Theorem 1 If the design is -optimal for the linear regression model (1), thenwhere the
An algorithm for construction of optimal designs
The fact that the optimality theorems only give a necessary condition for design optimality does not usually create additional problems for the algorithms of construction of designs. Numerical computation of optimal designs for the common linear regression model (1) with given correlation function can be performed by an extension of the multiplicative algorithm proposed by Dette et al. (2008c) for the case of non-correlated observations (see also Yu, 2010, for some extensions). Note that the
Examples
In this section, we provide some numerical results which we have obtained applying the algorithm described in Section 4 for the calculation of -optimal designs in several regression models. In the tables below we shall use the following notation for the -efficiency, -efficiency, and -efficiency of a design : and Here, is the -optimal design, is the -optimal design, is
Acknowledgments
The authors thank both referees for very valuable comments.
This work has been supported in part by the Collaborative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823, Teilprojekt C2) of the German Research Foundation (DFG). The work of Pepelyshev was partly supported by the Russian Foundation of Basic Research, project 12-01-00747. Parts of this paper were written while H. Dette was visiting the Institute of Mathematical Sciences at the National University of
References (28)
- et al.
Improving updating rules in multiplicative algorithms for computing -optimal designs
Computational Statistics and Data Analysis
(2008) Design of spatial experiments: model fitting and prediction
- et al.
Equidistant and -optimal designs for parameters of Ornstein–Uhlenbeck process
Statistics & Probability Letters
(2008) - et al.
Optimum experimental designs for properties of a compartmental model
Biometrics
(1993) - et al.
Experimental design and observation for large systems
Journal of the Royal Statistical Society, Series B
(1996) - et al.
Robustness of design against autocorrelation in time I: asymptotic theory, optimality for location and linear regression
Annals of Statistics
(1979) - et al.
Maximin optimal designs for a compartmental model. MODA 7
Advances in Model-oriented Design and Analysis
(2004) - et al.
Design of experiments in non-linear situations
Biometrika
(1959) - et al.
Bayesian experimental design: a review
Statistical Science
(1995) Locally optimal designs for estimating parameters
Annals of Mathematical Statistics
(1953)
Designing experiments with respect to standardized optimality criteria
Journal of the Royal Statistical Society, Series B
Optimal designs for dose-finding studies
Journal of the American Statistical Association
Exact optimal designs for weighted least squares analysis with correlated errors
Statistica Sinica
Optimality criteria for regression models based on predicted variance
Biometrika
Cited by (2)
Optimal designs for regression models with autoregressive errors
2016, Statistics and Probability LettersSpecial issue on algorithms for design of experiments
2014, Computational Statistics and Data Analysis