Weighted empirical likelihood estimates and their robustness properties

Abstract

Maximum likelihood methods are by far the most popular methods for deriving statistical estimators. However, parametric likelihoods require distributional specifications. The empirical likelihood is a nonparametric likelihood function that does not require such distributional assumptions, but is otherwise analogous to its parametric counterpart. Both likelihoods assume that the random variables are independent with a common distribution. A nonparametric likelihood function for data that are independent, but not necessarily identically distributed is introduced. The contaminated normal density is used to compare the robustness properties of weighted empirical likelihood estimators to empirical likelihood estimators. It is shown that as the contamination level of the sample increases, the root mean squared error of the empirical likelihood estimator for the mean increases. Conversely, the root mean squared error of the weighted empirical likelihood estimator for the mean remains closer to the theoretical root mean squared error.

Introduction

Likelihood-based methods are effective in finding efficient estimators, constructing tests with good power properties, and quantifying uncertainty through confidence intervals and confidence regions (Owen, 2001). Nevertheless, when the distribution is misspecified, parametric likelihood-based estimates are inefficient. In this case, the empirical likelihood is preferred because it does not require such distributional assumptions. Weighted empirical likelihood requires even fewer distributional assumptions since it relaxes the identical distributed assumption. Hence, it can offset problems arising from auxiliary information such as contamination by modifying the constraints or objective function. We develop weighted empirical likelihood for point estimators which extends to confidence intervals, confidence regions, and likelihood-based methods.

Empirical likelihood methods extend classical maximum likelihood methods for random samples from a common distribution of known functional form to the situation where the form of the distribution is unknown. In many applications, the observations may be from different distributions having a common mean. For example, if two types of measuring instruments have different variabilities and these instruments are used to obtain data, a fraction of these data comes from a more variable distribution. Classical robustness considered the problem of estimating the mean in one such case. In particular, Tukey (1960) introduced the contaminated normal family of densities $\mathit{CN}\left(\gamma ,{\sigma }^2\right)$:

$$f_{\gamma ,\sigma }(x)=\frac{1}{\sqrt{2\pi }}(1-\gamma ){\mathrm{e}}^{-x^2/2}+\frac{1}{\sigma \sqrt{2\pi }}\gamma {\mathrm{e}}^{-x^2/2{\sigma }^2}\text{,}$$

where $\gamma$, $0⩽\gamma ⩽1$ is the contamination parameter, and $\sigma$ is the scale parameter. The robustness implications of contaminated data were studied by Lehmann (1983), Gastwirth and Cohen (1970), Andrews and Mallows (1974), and more recently by Taskinen et al. (2003). They also arise in regression (Sinha and Wiens, 2002), and financial data (Ellis et al., 2003).

We compare the properties of the weighted empirical likelihood estimate to those of the usual empirical likelihood, the trimmed mean, and the Winsorized mean in the context of robust estimation. Results indicate that the weighted empirical likelihood estimate is somewhat more robust to contamination than the usual empirical likelihood estimate. As the sample size increases, weighted empirical likelihood estimators are comparable to the trimmed mean and the Winsorized mean. The trimmed mean and Winsorized mean's disadvantages are that the amount of trim and the number of observations replaced are somewhat arbitrary.

In Section 2, we review the empirical likelihood approach and introduce the weighted empirical likelihood for the mean. The major difference between weighted empirical likelihood and empirical likelihood is that the former incorporates a weight vector that weighs each observation's contribution to the likelihood function. Section 3 details how to obtain the weight vector, presents the nonlinear programming problem, then solves it to obtain the weighted empirical likelihood estimator for the mean. Section 4 explores the properties of the weighted empirical likelihood estimator when estimating the mean in the contaminated normal model. Concluding remarks are presented in Section 5.