A bias reducing technique in kernel distribution function estimation

Abstract

In this paper we suggest a bias reducing technique in kerneldistribution function estimation. In fact, it uses a convex combination of three kernel estimators, and it turned out that the bias has been reduced to the fourth power of the bandwidth, while the bias of the kernel distribution function estimator has the second power of the bandwidth. Also, the variance of the proposed estimator remains at the same order as the kernel distribution function estimator. Numerical results based on simulation studies show this phenomenon, too.

Appendix

Proof of Theorem 1

By a Taylor expansion

$$\begin{array}{l}\widehat{E F}(x+l h)\\ =\int\limits_{-\infty}^{\infty} W\left(\frac{x+l h-y}{h}\right) f(y) \mathrm{d} y\\ =\int\limits_{-\infty}^{x-(1-l) h} f(y) \mathrm{d} y+\int\limits_{x-(1-l) h}^{x+(1+l) h} W\left(\frac{x+l h-y}{h}\right) f(y) \mathrm{d} y\\ =F(x-(1-l) h)+\int\limits_{-1}^{1} W(t) f(x-(t-l) h) h \mathrm{d} t\\ =F(x)-(1-l) h f(x)+\frac{1}{2}(1-l)^{2} h^{2} f^{\prime}(x)-\frac{1}{6}(1-l)^{3} h^{3} f^{\prime \prime}(x)\\ +\frac{1}{24}(1-l)^{4} h^{4} f^{\prime \prime \prime}(x)+\int\limits_{-1}^{1} h W(t)\left\{f(x)-(t-l) h f^{\prime}(x)\right.\\ +\frac{1}{2}(t-l)^{2} h^{2} f^{\prime \prime}(x)-\frac{1}{6}(t-l)^{3} h^{3} f^{\prime \prime \prime}(x)+o\left(h^{3}\right) \} \mathrm{d} t+o\left(h^{4}\right), \end{array}$$

and by using the following facts;

$$\begin{array}{l}{\int(t-l) W(t) \mathrm{d} t=\left(1-\mu_{2}\right) / 2-l}, \\ {\int(t-l)^{2} W(t) \mathrm{d} t=\frac{1}{3}-\left(1-\mu_{2}\right) l+l^{2}}, \\ {\int(t-l)^{3} W(t) \mathrm{d} t=\left(1-\mu_{4}\right) / 4-l+\frac{3}{2} l^{2}\left(1-\mu_{2}\right)-l^{3}},\end{array}$$

we have

$$\begin{array}{l}{E \widehat{F}(x+l h)} \\ {=F(x)+l f(x) h+\frac{1}{2}\left(l^{2}+\mu_{2}\right) f^{\prime}(x) h^{2}+\frac{1}{6}\left(l^{3}+3 l \mu_{2}\right) f^{\prime \prime}(x) h^{3}} \\ {\quad+\frac{1}{24}\left(l^{4}+6 l^{2} \mu_{2}+\mu_{4}\right) f^{\prime \prime \prime}(x) h^{4}+o\left(h^{4}\right)}.\end{array}$$

Also, it is easy to show that

$$\begin{aligned} E\widehat{ f}(x+l h)=& f(x)+l f^{\prime}(x) h+\frac{1}{2}\left(l^{2}+\mu_{2}\right) f^{\prime \prime}(x) h^{2}+\frac{1}{6}\left(l^{3}+3 l \mu_{2}\right) f^{\prime \prime \prime}(x) h^{3} \\ &+\frac{1}{24}\left(l^{4}+6 l^{2} \mu_{2}+\mu_{4}\right) f^{(4)}(x) h^{4}+o\left(h^{4}\right). \end{aligned}$$

Hence,

$$\begin{aligned} E[\widehat{F}(x+l h)-lh \widehat{f}(x+l h)]=& F(x)+\frac{1}{2}\left(\mu_{2}-l^{2}\right) f^{\prime}(x) h^{2}-\frac{1}{3} l^{3} f^{\prime \prime}(x) h^{3} \\ &+\frac{1}{24}\left(\mu_{4}-3 l^{4}-6 l^{2} \mu_{2}\right) f^{\prime \prime \prime}(x) h^{4}+o\left(h^{4}\right). \end{aligned}$$

Therefore, $E \tilde{F}(x)$ can be computed by letting l = 0, l₁ and l₂, and the terms in h² and h³ vanish if and only if

$$\lambda_{1} l_{1}^{3}+\lambda_{2} l_{2}^{3}=0$$

and

$$\lambda_{1}\left(\mu_{2}-l_{1}^{2}\right)+\mu_{2}+\lambda_{2}\left(\mu_{2}-l_{2}^{2}\right)=0.$$

By letting −l₁ = l₂ = l, the two equations imply

$$l \equiv l(\lambda)=\sqrt{\frac{2 \lambda+1}{2 \lambda} \mu_{2}}.$$

Finally, if we substitute λ₁ = λ₂ = λ, −l₁ = l₂ = l = {μ₂(2λ + 1)/2λ}^1/2 for $E \tilde{F}(x)$, then we get the desired result for the bias part. Similar computations, though quite tedious, give the variance part. First, note that

$$\begin{aligned} \operatorname{Var}[\tilde{F}]=\frac{1}{(2 \lambda+1)^{2}} &\left[\lambda^{2} \operatorname{Var}\left(\widehat{F}_{1}\right)+\operatorname{Var}(\widehat{F})+\lambda^{2} \operatorname{Var}\left(\widehat{F}_{2}\right)\right.\\ &\left. +2 \lambda \operatorname{Cov}\left(\widehat{F}_{1}, \widehat{F}\right)+2 \lambda \operatorname{Cov}\left(\widehat{F}, \hat{F}_{2}\right)+2 \lambda^{2} \operatorname{Cov}\left(\widehat{F}_{1}, \widehat{F}_{2}\right)\right] . \end{aligned}$$

Now, we will compute each term on the right hand side of Var$[\tilde{F}]$ except $\operatorname{Var}(\widehat{F})$ which is given in Sect. 2. Let

$$W_{2}=\int\limits_{-1}^{1} W^{2}(t) \mathrm{d} t, \quad K_{2}=\int\limits_{-1}^{1} K^{2}(t) \mathrm{d} t,$$

then, for the first term, we have

$$\operatorname{Var}\left(\widehat{F}_{1}\right)=\operatorname{Var}[\widehat{F}(x-l h)]+l^{2} h^{2} \operatorname{Var} [\widehat{f}(x-l h) ]+2 lh \operatorname{Cov}[\widehat{F}(x-l h), \widehat{f}(x-l h)].$$

Now,

$$\begin{array}{l}{\operatorname{Var}[\widehat{F}(x-l h)]} \\ {\quad=\frac{1}{n}\left[E\left\{W^{2}\left(\frac{x-l h-y}{h}\right)\right\}-E^{2}\left\{W\left(\frac{x-l h-y}{h}\right)\right\}\right]} \\ {=\frac{1}{n} F(x)\{1-F(x)\}-\frac{h}{n} f(x)\left\{1+l-W_{2}-2 l F(x)\right\}+O\left(h^{2}\right)}\end{array}$$

because

$$E\left\{W^{2}\left(\frac{x-l h-y}{h}\right)\right\}=F(x)-(1+l) h f(x)+h f(x) W_{2}+O\left(h^{2}\right)$$

and

$$E^{2}\left\{W\left(\frac{x-lh-y}{h}\right)\right\}=F^{2}(x)-2 lhf(x) F(x)+O\left(h^{2}\right).$$

Also, it can be easily shown that

$$\operatorname{Var}[\widehat{f}(x-l h)]=\frac{1}{n h} f(x) K_{2}+O\left(h^{2}\right),$$

and

$$\operatorname{Cov}[\widehat{F}(x-l h), \widehat{f}(x-l h)]=\frac{1}{n}\left\{\frac{1}{2}-F(x)\right\}+O\left(h^{2}\right).$$

Therefore,

$$\operatorname{Var}\left(\widehat{F}_{1}\right)=\frac{1}{n} F(x)\{1-F(x)\}+\frac{h}{n} f(x)\left\{W_{2}+l^{2} K_{2}-1\right\}+O\left(h^{2}\right).$$

Similar computations give

$$\operatorname{Var}\left(\widehat{F}_{2}\right)=\frac{1}{n} F(x)\{1-F(x)\}+\frac{h}{n} f(x)\; \left\{W_{2}+l^{2} K_{2}-1\right\}+O\left(h^{2}\right)$$

since

$$\begin{aligned} \operatorname{Var}&[\widehat{F}(x+l h)] & \\=& \frac{1}{n}\left[E\left\{W^{2}\left(\frac{x+-y}{h}\right)\right\}-E^{2}\left\{W\left(\frac{x+l h-y}{h}\right)\right\}\right] \\=& \frac{1}{n} F(x)\{1-F(x)\}+\frac{h}{n} f(x)\left\{l-1+W_{2}-2 l F(x)\right\}+O\left(h^{2}\right). \end{aligned}$$

Now, we have

$$\begin{array}{l}{\operatorname{Cov}\left(\widehat{F_{1}}, \widehat{F}\right)} \\ {\;\;=-\frac{1}{n} F^{2}(x)+\frac{h}{n} f(x)\left\{\int W(t-l) W(t) \mathrm{d} t+\frac{l}{2} \int K(t-l) W(t) \mathrm{d} t\right\}+O\left(h^{2}\right)},\end{array}$$

$$\begin{array}{l}{\operatorname{Cov}\left(\widehat{F_{2}}, \widehat{F}\right)} \\ {\;\;=-\frac{1}{n} F^{2}(x)+\frac{h}{n} f(x)\left\{\int W(t+l) W(t) \mathrm{d} t-\frac{l}{2} \int K(t+l) W(t) \mathrm{d} t\right\}+O\left(h^{2}\right)},\end{array}$$

and

$$\begin{array}{l}{\operatorname{Cov}\left(\widehat{F}_{1}, \widehat{F}_{2}\right)} \\ {\quad=-\frac{1}{n} F^{2}(x)+\frac{h}{n} f(x)\left\{\int W(t-l) W(t+l) \mathrm{d} t+l \int K(t-l) W(t+l) d t\right.} \\ {\quad-l \int W(t-l) K(t+l) \mathrm{d} t-l^{2} \int K(t-l) K(t+l) \mathrm{d} t \}+O\left(h^{2}\right)}.\end{array}$$

By adding up all these terms we have the desired result for the variance.