Two generalized nonparametric methods for estimating like densities

Abstract

This article presents two generalized nonparametric methods for estimating multiple, possibly like, densities. The first generalization contains the Nadaraya–Watson estimator, the Jones et al. (Biometrika 82(2):327–338, 1995) bias reduction estimator, and Ker (Stat Probab Lett 117:23–30, 2016) possibly similar estimator as special cases. The second generalization contains the Nadaraya–Watson estimator, Ker (2016) possibly similar estimator, and the conditional density estimator of Hall et al. (J Am Stat Assoc 99(468):1015–1026, 2004) as special cases. The generalizations do not require knowledge of the form or extent of likeness between the unknown densities; an attractive feature in empirical applications. Numerical simulations demonstrate that the two proposed generalizations lead to significant efficiency gains.

Appendix: proofs

Throughout the proofs, we let $X_l^j$ denote the realizations from the unit of interest while $X_l^{-j}$ denote the realizations from the other or extraneous units.

Theorem 1

The NW, JLN and Ker estimators are special cases of the G1 estimator.

Lemma 1

If $h_p\rightarrow \infty$, then $\hat{f}_{G1}(x)=\hat{f}_{NW}(x)$.

Proof

If $h_p\rightarrow \infty$, then

$$\begin{aligned} \hat{g}_{G1}(x) =\frac{1}{n}\sum _{l=1}^{nQ} K^d(l,j)K_{h_p}\left( x-X_l\right) \approx \frac{1}{n}\sum _{l=1}^{nQ} K^d(l,j)K_{h_p}(0), \end{aligned}$$

and

$$\begin{aligned} \hat{g}_{G1}(X_l^{j}) =\frac{1}{n}\sum _{l=1}^{nQ} K^d(l,j)K_{h_p}\left( x-X_l^{j}\right) \approx \frac{1}{n}\sum _{l=1}^{nQ} K^d(l,j)K_{h_p}(0), \end{aligned}$$

thus, $\hat{g}_{G1}(x)$ is approximately uniform. Hence,

$$\begin{aligned} \hat{f}_{G1}(x) =\frac{1}{n}\sum _{l=1}^{n} K_{h_{G1}}\left( x-X_l^j \right) = \hat{f}_{NW} \end{aligned}$$

$\square$

Lemma 2

If $\lambda =0$ then $\hat{f}_{G1}(x)=\hat{f}_{JLN}(x)$.

Proof

If $\lambda =0$, then $K^d(l,j)=0^{I(l,j)}1^{1-I(l,j)}= 1$ if $l=j$ and 0 otherwise.

Note,

$$\begin{aligned} \hat{g}_{G1}(x)&=\frac{1}{n}\left[ \sum _{l=1}^{n} 1\times K_{h_p}\left( x-X_l^j\right) +\sum _{l=1}^{n(Q-1)} 0\times K_{h_p}\left( x-X_l^{-j}\right) \right] \\&=\frac{1}{n} \sum _{l=1}^{n} K_{h_p}\left( x-X_l^j\right) . \end{aligned}$$

and thus

$$\begin{aligned} \hat{f}_{G1}(x)= \frac{1}{n} \sum _{l=1}^{n} \frac{K_{h_{G1}}(x-X_l^j)}{\hat{f}(X_l^{j})} \hat{f}(x) = \hat{f}_{JLN}(x). \end{aligned}$$

$\square$

Lemma 3

If $\lambda =\frac{Q-1}{Q}$ then $\hat{f}_{G1}(x)=\hat{f}_K(x)$.

Proof

If $\lambda =\frac{Q-1}{Q}$, then $\frac{\lambda }{Q-1}=1-\lambda =\frac{1}{Q}$, $K^d(l,j)=\frac{1}{Q}^{I(l,j)} \frac{1}{Q}^{1-I(l,j)}= \frac{1}{Q} \; \forall \; l,j$ thus

$$\begin{aligned} \hat{g}_{G1}(x)&=\frac{1}{n}\left[ \sum _{l=1}^{n} \frac{1}{Q}\times K_{h_p}\left( x-X_l^{j}\right) +\sum _{l=1}^{n(Q-1)} \frac{1}{Q}\times K_{h_p}\left( x-X_l^{-j}\right) \right] \\&=\frac{1}{nQ} \sum _{l=1}^{nQ} K_{h_p}\left( x-X_l\right) = \hat{g}(x) . \end{aligned}$$

and thus

$$\begin{aligned} \hat{f}_{G1}(x)= \frac{1}{n} \sum _{l=1}^{n} \frac{K_{h_{G1}}(x-X_l^j)}{\hat{g}(X_l^{j})} \hat{g}(x) = \hat{f}_K(x). \end{aligned}$$

$\square$

Theorem 2

The NW, HRL and Ker estimators are special cases of the G2 estimator.

Lemma 4

If $h_p \rightarrow \infty$, $\lambda =0$ then $\hat{f}_{G2}(x)=\hat{f}_{NW}(x)$.

Proof

If $h_g \rightarrow \infty$, then

$$\begin{aligned} \hat{g}_{G2}(x) =\frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}\left( x-X_l\right) \approx \frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}(0), \end{aligned}$$

and

$$\begin{aligned} \hat{g}_{G2}(X_i^j) =\frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}\left( X_l-X_i^j\right) \approx \frac{1}{nQ}\sum _{p=1}^{nQ} K_{h_p}(0), \end{aligned}$$

thus $\hat{g}_{G2}(x)$ and is approximately uniform. Then

$$\begin{aligned} \hat{f}_{G2}(j,x)=\frac{1}{nQ}\sum _{l=1}^{nQ}K^d(l,j)K_{h_{G2}}(x-X_l) \end{aligned}$$

and

$$\begin{aligned} \hat{f}_{G2}(x)=\frac{\hat{f}_{G2}(j,x)}{\hat{m}(j)} =\frac{1}{n}\sum _{l=1}^{nQ}K^d(l,j)K_{h_{G2}}(x-X_l). \end{aligned}$$

If $\lambda =0$, then $K^d(l,j)=0^{I(l,j)} 1^{1-I(l,j)} = 1$ if $l=j$ and 0 otherwise.

$$\begin{aligned} \hat{f}_{G2}(x)&=\frac{1}{n} \left[ \sum _{l=1}^{n} 1\times K_{h_{G2}}\left( x-X_l^j\right) +\sum _{l=1}^{n(Q-1)} 0\times K_{h_{G2}}\left( x-X_l^{-j}\right) \right] \\&=\frac{1}{n} \sum _{l=1}^{n} K_{h_{G2}}\left( x-X_l^j\right) = \hat{f}_{NW}(x). \end{aligned}$$

$\square$

Lemma 5

If $h_p \rightarrow \infty$ then $\hat{f}_{G2}(x)=\hat{f}_{HRL}(x)$.

Proof

If $h_p \rightarrow \infty$, then

$$\begin{aligned} \hat{g}_{G2}(x) =\frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}(x-X_l) \approx \frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}(0), \end{aligned}$$

and

$$\begin{aligned} \hat{g}_{G2}(X_i^j) =\frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}(X_l-X_i^j) \approx \frac{1}{nQ}\sum _{l=1}^{nQ} K_{h_p}(0), \end{aligned}$$

thus $\hat{g}_{G2}(x)$ is approximately uniform then

$$\begin{aligned} \hat{f}_{G2}(j,x)&=\frac{1}{nQ}\sum _{l=1}^{nQ}K^d(l,j)L(x,X_l)\\&=\frac{1}{nQ}\sum _{l=1}^{nQ}K^d(l,j)K_{h_{G2}} (x-X_l) \end{aligned}$$

and

$$\begin{aligned} \hat{f}_{G2}(x)=\frac{\hat{f}_{G2}(j,x)}{\hat{m}(j)} =\frac{1}{n}\sum _{l=1}^{nQ}K^d(l,j)K_{h_{G2}}(x-X_l) = \hat{f}_{HRL}(x). \end{aligned}$$

$\square$

Lemma 6

If $\lambda =0$ then $\hat{f}_{G2}(x)=\hat{f}_{K}(x)$.

Proof

If $\lambda =0$, then $K^d(l,j)=0^{I(l,j)} 1^{1-I(l,j)} = 1$ if $l = j$ and 0 otherwise.

$$\begin{aligned} \hat{f}_{G2}(j,x)&= \frac{1}{nQ}\left[ \sum _{l=1}^{n} 1\times \frac{ K_{h_{G2}}(x-X_l^{j})}{\hat{g}_{G2}(X_l^{j})} \hat{g}_{G2}(x) +\sum _{l=1}^{n(Q-1)} 0\times \frac{ K_{h_{G2}}(x-X_l^{-j})}{\hat{g}_{G2}(X_l^{-j})} \hat{g}_{G2}(x) \right] \\&= \frac{1}{nQ} \sum _{l=1}^{n} \frac{ K_{h_{G2}}(x-X_l^{j})}{\hat{g}_{G2}(X_l^{j})} \hat{g}_{G2}(x), \end{aligned}$$

and then

$$\begin{aligned} \hat{f}_{G2}(x) = \frac{1}{n} \sum _{l=1}^{n} \frac{ K_{h_{G2}}(x-X_l^j)}{\hat{g}_{G2}(X_l^{j})} \hat{g}_{G2}(x) = \hat{f}_K(x). \end{aligned}$$

$\square$