Semiparametric estimation of the link function in binary-choice single-index models

Abstract

We propose a new, easy to implement, semiparametric estimator for binary-choice single-index models which uses parametric information in the form of a known link (probability) function and nonparametrically corrects it. Asymptotic properties are derived and the finite sample performance of the proposed estimator is compared to those of the parametric probit and semiparametric single-index model estimators of Ichimura (J Econ 58:71–120, 1993) and Klein and Spady (Econometrica 61:387–421, 1993). Results indicate that if the parametric start is correct, the proposed estimator achieves significant bias reduction and efficiency gains compared to Ichimura (1993) and Klein and Spady (1993). Interestingly, the proposed estimator still achieves significant bias reduction and efficiency gains even if the parametric start is not correct.

Appendix

We make the following assumptions:

Assumption 1:

Observed sample $(x_i,y_i)$, $i=1,\ldots ,n$ is i.i.d.

Assumption 2:

$B\subset {\mathbb {R}}^q$ is compact and the true parameter vector $b_0$ is in the interior of B.

Assumption 3:

$A_x$ is compact.

Assumption 4:

K(s) is a density. Furthermore $\int sK(s)ds=0$, $K(s)=0$ for $s<-1$ and $s>1$, and its second derivative satisfies a Lipschitz condition.

Assumption 5:

Parametric start G is uniformly bounded over x, b and $G(xb)\ne 0\; \forall x,b\in A_x\times B$.

Assumption 6:

$\int |\phi (t)|dt < \infty$ where $\phi (t)$ is the characteristic function of K.

Assumption 7:

There exist ${\underline{F}}$ and ${\overline{F}}$ that do not depend on x such that $0<{\underline{F}}\leqslant F(xb) \leqslant {\overline{F}}~<~1 \quad \forall b \in B$.

Proof of lemma 1

Our proof of lemma 1 follows closely Bierens (1987a, b) and Pagan and Ullah (1999, pp. 36–39). Note that ${\hat{F}}$ in (9) can be written as ${\hat{F}}_{1} / {\hat{F}}_{2}$ where

$$\begin{aligned} {\hat{F}}_{1}&= \frac{1}{(n-1)h} \sum _{j\ne i}y_j{\mathbf {1}}_{[x_j\in A_{nx}]}\bigg \{\frac{G(x_{i}b)}{G(x_{j}b)} \bigg \} K\left( \frac{x_{i}b-x_{j}b}{h}\right) \\ {\hat{F}}_{2}&= \frac{1}{(n-1)h} \sum _{j\ne i}{\mathbf {1}}_{[x_j\in A_{nx}]}K\left( \frac{x_{i}b-x_{j}b}{h}\right) . \end{aligned}$$

Since ${\hat{F}}_{2}$ is ${\hat{F}}_{1}$ with $y_j=1$ and $G(\cdot )$ a constant function, we will only show uniform convergence of ${\hat{F}}_{1}$.²² Now observe that

$$\begin{aligned} E(y|x)=F(x)=\frac{G(x)\int y \frac{1}{G(x)}f(y,x)dy}{\int f(y,x)dy}. \end{aligned}$$

Let $g(x)=\int y \frac{1}{G(x)} f(y,x)dy / \int f(y,x)dy$ and $h(x)=\int f(y,x)dy$. So $F(x)=G(x)g(x)$ and $G(x)g(x)h(x)=G(x)\int y\frac{1}{G(x)}f(y,x)dy$. Thus ${\hat{F}}_{1}=G(x){\hat{g}}(x){\hat{h}}(x)$. Notice that by assumption 5, G is uniformly bounded and we have $\sup _x |G(x)|=O(1)$. In fact a plausible start is a distribution function in which case $\sup _x |G(x)|=1$. Hence it suffices to show $\sup _x |{\hat{g}}(x){\hat{h}}(x) - g(x)h(x)|\rightarrow 0$. So we have

$$\begin{aligned}&\sup _x |{\hat{g}}(x){\hat{h}}(x) - g(x)h(x)|\leqslant \sup _x |{\hat{g}}(x){\hat{h}}(x) - E{\hat{g}}(x){\hat{h}}(x)|\nonumber \\&\quad + \sup _x |E{\hat{g}}(x){\hat{h}}(x)-g(x)h(x)|. \end{aligned}$$

(11)

Like Ichimura (1993), we will refer to the second term of the right-hand side as bias term and show that it converges to 0 at the rate $h^2$. But first notice that

$$\begin{aligned} {\hat{g}}(x){\hat{h}}(x)=\frac{1}{nh}\sum _{j=1}^n\frac{y_j}{G(x_j)}K\left( \frac{x-x_j}{h}\right) . \end{aligned}$$

(12)

From the inversion formula (see Fristedt et al. 1997, p. 231) and by assumption 6 we have

$$\begin{aligned} K(a)=\frac{1}{2\pi }\int \exp (-ita)\phi (t)dt \end{aligned}$$

(13)

where $\phi (t)$ is the characteristic function of K and $i^2=-1$. Using (12) and (13) and letting $s=t/h$ we get

$$\begin{aligned} {\hat{g}}(x){\hat{h}}(x)=\frac{1}{2\pi }\int \bigg \{ \frac{1}{n}\sum _{j=1}^{n}\frac{y_j}{G(x_j)} \exp (itx_j) \bigg \} \exp (-itx) \phi (ht)dt. \end{aligned}$$

(14)

From (14) we get

$$\begin{aligned} E{\hat{g}}(x){\hat{h}}(x)=\frac{1}{2\pi }\int E\bigg [ \frac{y_j}{G(x_j)}\exp (itx_j) \bigg ] \exp (-itx) \phi (ht)dt. \end{aligned}$$

(15)

From (14) and (15) and noting that $|\exp (-itx)|=1$

$$\begin{aligned}&|{\hat{g}}(x){\hat{h}}(x)- E{\hat{g}}(x){\hat{h}}(x)| \leqslant \frac{1}{2\pi } \int \left| \frac{1}{n} \sum _{j=1}^n \left\{ \frac{y_j}{G(x_j)}\exp (itx_j)\right. \right. \\&\quad \left. \left. - E\left[ \frac{y_j}{G(x_j)}\exp (itx_j)\right] \right\} \right| |\phi (ht)|dt. \end{aligned}$$

So

$$\begin{aligned}&\sup _{x}|{\hat{g}}(x){\hat{h}}(x)- E{\hat{g}}(x){\hat{h}}(x)| \\&\quad \leqslant \frac{1}{2\pi } \int \left| \frac{1}{n} \sum _{j=1}^n \left\{ \frac{y_j}{G(x_j)}\exp (itx_j) - E\left[ \frac{y_j}{G(x_j)}\exp (itx_j)\right] \right\} \right| |\phi (ht)|dt. \end{aligned}$$

Using $\exp (itx_j)=\cos (tx_j)+i\sin (txj)$ we can write

$$\begin{aligned} \begin{aligned}&E\left| \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\exp (itx_j) -E\left[ \frac{y_j}{G(x_j)}\exp (itx_j)\right] \right\} \right| \\&\quad =E\Bigg | \underset{A}{ \underbrace{ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\cos tx_j -E\left[ \frac{y_j}{G(x_j)}\cos tx_j\right] \right\} }} \\&\qquad +\,i \underset{B}{ \underbrace{ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\sin tx_j -E\left[ \frac{y_j}{G(x_j)}\sin tx_j\right] \right\} }} \Bigg | \end{aligned} \end{aligned}$$

(16)

Note that we can write $|A+iB|=(A^2+B^2)^{1/2}$ and so $E|A+iB|=E(A^2+B^2)^{1/2}\leqslant (EA^2+EB^2)^{1/2}=(Var(A)+Var(B))^{1/2}$ where the inequality comes from Jensen’s inequality and by construction $EA=0$ and $EB=0$. So (16) is

$$\begin{aligned}&\leqslant \left\{ Var\left[ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\cos tx_j -E\left[ \frac{y_j}{G(x_j)}\cos tx_j\right] \right\} \right] \right. \\&\qquad \left. + Var\left[ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\sin tx_j -E\left[ \frac{y_j}{G(x_j)}\sin tx_j\right] \right\} \right] \right\} ^{1/2} \\&\quad = \left\{ Var\left[ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\cos tx_j \right\} \right] + Var\left[ \frac{1}{n}\sum _{j=1}^n\left\{ \frac{y_j}{G(x_j)}\sin tx_j \right\} \right] \right\} ^{1/2}\\&\quad = \left\{ \frac{1}{n} \left( Var\left[ \frac{y_j}{G(x_j)}\cos tx_j \right] + Var\left[ \frac{y_j}{G(x_j)}\sin tx_j \right] \right) \right\} ^{1/2}. \end{aligned}$$

Note that $VarX\leqslant EX^2$ so

$$\begin{aligned}&\leqslant \left\{ \frac{1}{n} \left( E\left[ \left( \frac{y_j}{G(x_j)}\right) ^2 \cos ^2 tx_j \right] + E\left[ \left( \frac{y_j}{G(x_j)}\right) ^2 \sin ^2 tx_j \right] \right) \right\} ^{1/2} \\&= \frac{1}{\sqrt{n}} \left\{ E\left[ \left( \frac{y_j}{G(x_j)} \right) ^2 \right] \right\} ^{1/2} \end{aligned}$$

noting that $\cos ^2 tx_j+\sin ^2 tx_j = 1$. So

$$\begin{aligned} E\sup _x |{\hat{g}}(x){\hat{h}}(x)- E{\hat{g}}(x){\hat{h}}(x)|&\leqslant \frac{1}{2\pi }\frac{1}{\sqrt{n}} \left\{ E\left[ \left( \frac{y_j}{G(x_j)}\right) ^2\right] \right\} ^{1/2} \int |\phi (ht)|dt \nonumber \\&= \frac{1}{2\pi }\frac{1}{h\sqrt{n}} \left\{ E\left[ \left( \frac{y_j}{G(x_j)}\right) ^2\right] \right\} ^{1/2} \int |\phi (s)|ds \end{aligned}$$

(17)

after a change of variables ($s=ht$) and the last term goes to zero as $h\sqrt{n}\rightarrow \infty$. Finally using Markov’s inequality with (17) we get

$$\begin{aligned} Pr\left( \sup _x |{\hat{g}}(x){\hat{h}}(x)- E{\hat{g}}(x){\hat{h}}(x)| > \epsilon \right) \rightarrow 0 \quad \text {as} \quad n\rightarrow \infty . \end{aligned}$$

Now for $|E{\hat{g}}(x){\hat{h}}(x)- g(x)h(x)|$ note that

$$\begin{aligned} E{\hat{g}}(z){\hat{h}}(z)&= E\left[ \frac{1}{nh}\sum _{j=1}^n y_j\frac{1}{G(x_j)} K\left( \frac{z-x_j}{h}\right) \right] \nonumber \\&= \frac{1}{h}E\left[ y_j\frac{1}{G(x_j)} K\left( \frac{z-x_j}{h}\right) \right] \end{aligned}$$

(18)

$$\begin{aligned}&= \frac{1}{h} \int K\left( \frac{z-x}{h}\right) \underbrace{\int y \frac{1}{G(x)}f(y,x)dy}_{g(x)h(x)} dx. \end{aligned}$$

(19)

Now let $\psi (x)=g(x)h(x)$ and $s=(z-x)/h$ for the Taylor expansion

$$\begin{aligned} \psi (x)=\psi (z-sh)=\psi (z)-hs\psi '(z)+\frac{1}{2}h^2 s^2 \psi ''(z) + o(h^2). \end{aligned}$$

So

$$\begin{aligned} \frac{1}{h} \int \left( \psi (z)-hs\psi '(z)+\frac{1}{2}h^2 s^2 \psi ''(z) \right) K(s)hds =\psi (z)+\frac{1}{2}h^2 \psi ''(z)\int s^2 K(s)ds. \end{aligned}$$

Thus we can write

$$\begin{aligned} \sup _x\left| E{\hat{\psi }}(x)-\psi (x)\right|&= \sup _x\left| \psi (x)+\frac{1}{2}h^2 \psi ''(x) \int s^2 K(s)ds - \psi (x) \right| \\&\leqslant \frac{1}{2}h^2 \sup _x|\psi ''(x)|\int |s^2 K(s)|ds \end{aligned}$$

so the last term goes to zero at the rate $h^2$. This completes the proof of lemma 1.

Proof of theorem 1

Let z represent a point on the support of $z_i$. Given the binary nature of the dependent variable $y_i$ and the fact that $P(y_i=1)=F(z_i)$, we have $E(y_i|z_i)=F(z_i)$. Hence: $y_i=F(z_i)+\epsilon _i$ where $\epsilon$ is an i.i.d error term such that $E(\epsilon _i|z_i=0)$ and $Var(\epsilon _i|z_i)=F(z_i)(1-F(z_i))~~\forall i.$ In deriving the asymptotic properties of the proposed link function estimator (PGSIM), we require, in addition to Assumptions 1 and 5, the following assumptions:

Assumption 8:

The density function of the index f(z) with bounded support Z, and the unknown link function F(z) $\in {{{\mathcal {C}}}}^{2}(\Theta )$ with finite second derivatives and $f(z)\ne 0$ in $\Theta$, the neighborhood of point z.

Assumption 9:

The Kernel function K(s) is bounded, real-valued, with the following characteristics: (i) $\int K(s)ds=1$, (ii) K(s) is symmetric about 0, (iii) $\int s^{2}K(s)ds <\infty$, (iv) $\vert s\vert K(\vert s \vert )\rightarrow 0$ as $\vert s \vert \rightarrow \infty$, (v) $\int K^{2}(s)ds \le \infty .$

Assumption 10:

$h\rightarrow 0$ and $nh\rightarrow \infty$.

Assumption 11:

$E\left| \frac{G(z)}{G(z_i)}\right| ^{2+\delta },~~E\vert \epsilon _{i}\vert ^{2+\delta }$, and $\int \vert K(\omega )\vert ^{2+\delta }$ are finite for some $\delta > 0.$

The nonparametric Kernel estimator of the link function (Ichimura 1993; Klein and Spady 1993) is:

$$\begin{aligned} {\tilde{F}}(z)=\frac{\sum _iy_{i}K_{h}(z_{i}-z)}{\sum _iK_{h}(z_{i}-z)} \end{aligned}$$

(20)

Denoting $\mu _{2}=\int s^{2}K(s)ds$ and $\text{ R }(K)= \int K^{2}(s) dz$, standard properties of ${\tilde{F}}(z)$ are:

$$\begin{aligned} E({\tilde{F}}(x)-F(z))= & {} \frac{1}{2}\mu _{2}h^{2}\left( F^{\prime \prime }(z)+2F^{\prime }(z)\frac{f^{\prime }(z)}{f(z)}\right) +o_p(h^{2}) , \end{aligned}$$

(21)

$$\begin{aligned} \text{ V }ar({\tilde{F}}(z))= & {} \frac{\sigma ^{2}(z) R(K)}{(nh)f(z)}+O_p(h/n) \end{aligned}$$

(22)

where $\sigma ^2(z)=F(z)(1-F(z))$.

Now consider our parametrically guided nonparametric estimator of the link function:

$$\begin{aligned} {\hat{F}}(z)= & {} \frac{\sum _{i=1}^{n}y_i\left[ \frac{G(z)}{G(z_i)}\right] K_{h}(z_i-z)}{\sum _{i=1}^{n}K_{h}(z_{i}-z)}\end{aligned}$$

(23)

$$\begin{aligned}= & {} \frac{n^{-1}\sum _{i=1}^{n}y_i\left[ \frac{G(z)}{G(z_i)}\right] K_{h}(z_i-z)}{{\hat{f}}(z)} \end{aligned}$$

(24)

First, we derive the bias and variance of the proposed estimator. We have

$$\begin{aligned} \left( \hat{F(z)}-F(z)\right) {\hat{f}}(z)= & {} n^{-1}\sum _i K_h(z_i-z)\left[ \frac{G(z)}{G(z_i)}\right] y_i - n^{-1}\sum _i K_h(z_i-z)F(z)\\= & {} n^{-1}\sum _i K_h(z_i-z)\left( \left[ \frac{G(z)}{G(z_i)}\right] y_i-F(z)\right) \\= & {} n^{-1}\sum _i K_h(z_i-z)\left( G(z)\frac{F(z_i)+\epsilon _i}{G(z_i)}-r(z)G(z)\right) \\= & {} n^{-1}G(z)\sum _i K_h(z_i-z)\left( r(z_i)-r(z)\right) ~~~~~~~~\left( \hbox { }\ C_n\right) \\+ & {} n^{-1}G(z)\sum _i K_h(z_i-z)\frac{\epsilon _i}{G(z_i)}~~~~~~~~~~~~~~~~~~~~~~\left( \hbox { }\ D_n\right) \end{aligned}$$

Per assumption 1, we have,

$$\begin{aligned} E(C_n)= & {} G(z)\int K_{h}(z_{1}-z)\left( r(z_1)-r(z)\right) f(z_{1})dz_{1} \\= & {} G(z)\int K(s)\left( r(z+hs)-r(z)\right) f(z+hs)ds \\&\text{ after } \text{ a } \text{ change } \text{ of } \text{ variable }\\= & {} \frac{h^{2}}{2}\left( G(z)f(z)r^{\prime \prime }(z)+2G(z)f{\prime }(z)r^{\prime }(z)\right) \mu _{2}(K)+o_p(h^{2}). \end{aligned}$$

It can be easily seen that $E(D_n)=0$. Using the fact that ${\hat{f}}(z)=f(z)+o_p(1)$, we obtain

$$\begin{aligned} E({\hat{F}}(z)-F(z))= & {} \frac{1}{2}\mu _{2}h^{2}\left( r^{\prime \prime }(z)+2r^{\prime }(z)\frac{f^{\prime }(z)}{f(z)}\right) G(z)+o_p(h^{2}) \end{aligned}$$

(25)

Turning to the variance of the estimator, we have

$$\begin{aligned} Var(C_n)= & {} Var\left[ n^{-1}G(z)\sum _i^{n}K_{h}(z_{i}-z)(r(z_i)-r(z))\right] \\= & {} (n)^{-2}G^2(z)\left( E\left[ \sum _i^{n}K^2_{h}(z_{i}-z)(r(z_{i}-r(z))^2\right] \right. \\&\left. - \left[ E\sum _i^{n}K_{h}(z_{i}-z)(r(z_{i})-r(z))\right] ^{2}\right) \\= & {} (nh)^{-1} G^2(z) \int K^{2}(s)(r(z+hs)-r(z))^2 f(z+hs)ds \\&+\frac{n(n-1)}{n^{2}} G^2(z)\left[ \int K(s)(r(z+hs)-r(z))f(z+hs)ds\right] ^{2} \\&- G^2(z)\left[ \int K(s)(r(z+hs)-r(z))f(z+hs)ds\right] ^{2}\\= & {} (nh)^{-1} \int K^{2}(s) [hs~r^{\prime }(z)]^2 f(z)ds + O(n^{-1})+ o_p((nh)^{-1}) \\ \end{aligned}$$

Hence $Var(C_n)=o_p((nh)^{-1})$.

$$\begin{aligned} Var(D_n)= & {} E(Var_{z_i}(D_n))~~\hbox {since} E_{z_i}(D_n)=0 \hbox {, where} Var_{z_{i}}\,\\&\hbox {denotes the conditional variance of} D_n\\= & {} E\left[ n^{-2}G(z)^2 \sum _i K^2_h(z_i-z)G^{-2}(z_{i})\sigma ^2(z_i)\right] \\= & {} (nh)^{-1}\sigma ^2(z)G(z)^2\int K^{2}(s)ds[ G(z)^{-2}(z)f(z) ]+o((nh)^{-1})\\= & {} (nh)^{-1}\sigma ^2(z)R(K)f(z)+o_p((nh)^{-1}) \end{aligned}$$

Finally,

$$\begin{aligned} Cov(C_n,D_n)= & {} E(C_nD_n)~~\hbox { since}\ E(D_n)=0\\= & {} (n)^{-2}G^2(z)E \left[ \sum _i K_h^{2}(z_i-z)(r(z_{i})-r(z))^2\epsilon (z_i)^2\right] \\&+\frac{n(n-1)}{(n)^{2}}G^2(z)E\left[ K_h(z_{1}-z)(r(z_{1}-r(z))\right] E\left[ K_h(z_{1}-z)\epsilon (z_1)\right] \\= & {} (nh)^{-1} G^2(z)\sigma ^2(z)h^2r^{\prime }(z)f(z) \int s^2 K^{2}(s) ds\\= & {} o_p((nh)^{-1}) \end{aligned}$$

Piecing the results together, we have

$$\begin{aligned} \text{ V }ar({\hat{F}}(z))= & {} \frac{\sigma ^{2}(z) R(K)}{(nh)f(z)}+o_p(nh)^{-1}. \end{aligned}$$

Note that the variance function is the same as the variance of the Klein and Spady or Ichimura estimator.

Next, we show that the proposed semiparametric link function estimator ${\hat{F}}(z)$ has a limiting normal distribution

$$\begin{aligned} \sqrt{nh}({\hat{F}}(z)-F(z)) \rightarrow {\mathcal {N}}(B(h),\Sigma ) \end{aligned}$$

(26)

where $B(h)=\frac{1}{2}\mu _{2}h^{2}\left( r^{\prime \prime }(z)+2r^{\prime }(z)\frac{f^{\prime }(z)}{f(z)}\right) G(z)$ and $\Sigma =\frac{\sigma ^{2}(z)}{f(z)}R(K)$.

From the results above, it can be seen that

$$\begin{aligned} C_{n}= & {} f(z)B(h)+o_{p}(h^{2}); \end{aligned}$$

We have also shown that $E(D_{n})=0$ and $Var(D_{n})=(nh)^{-1}\left( \sigma ^{2}(z)R(K)f(z)+o(1)\right) .$ $D_{n}$ is a triangular array of i.i.d. random variables; thus, under assumption A6,

$$\begin{aligned}(nh)^{-\delta /2}E\left| \frac{G(z)}{G(z_i)}\right| ^{2+\delta }E\left| \epsilon _i\right| ^{2+\delta }E\left| K_h(z_i-z)\right| ^{2+\delta }h^{-1}\rightarrow 0 \text{ as } n \rightarrow \infty . \end{aligned}$$

Hence we can apply Liapounov’s central limit theorem to obtain $\sqrt{nh} (D_{n}) \rightarrow {\mathcal {N}}(0,f^{2}(x)\Sigma )$. Since $plim {\hat{f}}(z)=f(z)$, it also follows that

$$\begin{aligned} \sqrt{nh}({\hat{F}}(z)-F(z)-B(h))= \sqrt{nh}\frac{D_{n}}{f(x)}+o_{p}(1) \rightarrow {\mathcal {N}}(0,\Sigma ). \end{aligned}$$

(27)

Proof of theorem 2

Note that

$$\begin{aligned} \sup _b|{\hat{Q}}_n(b) - Q_0(b) | \leqslant \sup _b |{\hat{Q}}_n(b) - Q_n(b) |+ \sup _b |Q_n(b) - Q_0(b) | \end{aligned}$$

where

$$\begin{aligned} {\hat{Q}}_n(b)&= \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}(y_{i}\log [{\hat{F}}(x_i b)]+(1-y_i)\log [1-{\hat{F}}(x_i b)]), \\ Q_n(b)&= \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}(y_{i}\log [F(x_i b)]+(1-y_i)\log [1-F(x_i b)]), \\ Q_0(b)&= \frac{1}{n}\sum _{i=1}^{n} E\left[ {\mathbf {1}}_{[x_i\in A_x]}(y_{i}\log [F(x_i b)]+(1-y_i)\log [1-F(x_i b)]) \right] . \\ \end{aligned}$$

Let ${\hat{Q}}_{1n}=n^{-1}\sum _{i=1}^n {\mathbf {1}}_{[x_i\in A_x]}y_i \log [{\hat{F}}(x_i b)]$ and similarly for $Q_{1n}$. Let ${\hat{F}}_i\equiv {\hat{F}}(x_ib)$ and similarly for $F_i$. ${\hat{Q}}_{1n}$ can be viewed as a function of ${\hat{F}}_i$, so from a Taylor expansion of ${\hat{Q}}_{1n}$ about $F_i$ we get

$$\begin{aligned} |{\hat{Q}}_{1n}-Q_{1n}|&=\left| \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}y_{i}\log [F_i]\right. \\&\quad \left. + \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}y_{i}\frac{1}{\tilde{F_i}}({\hat{F}}_i-F_i)- \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}y_{i}\log [F_i] \right| \\&\leqslant \frac{1}{n}\sum _{i=1}^{n}{\mathbf {1}}_{[x_i\in A_x]}y_{i}\frac{1}{\tilde{F_i}}|{\hat{F}}_i-F_i| \end{aligned}$$

where ${\tilde{F}}_i$ is between ${\hat{F}}_i$ and $F_i$. So we have

$$\begin{aligned} \sup _b |{\hat{Q}}_{1n}-Q_{1n}| \leqslant \sup _{i,b}|q_i| \frac{1}{n}\sum _{i=1}^{n}\sup _b|{\hat{F}}_i-F_i| \end{aligned}$$

where $q_i={\mathbf {1}}_{[x_i\in A_x]}y_i / {\tilde{F}}_i$. Note that $\sup _{i,b}|q_i|=O(1)$ and from lemma 1 $\sup _b|{\hat{F}}_i-F_i|\rightarrow 0$ in probability so $\sup _b |{\hat{Q}}_{1n}-Q_{1n}|=o_p(1)$. A similar result can be obtained for $y_i=0$ part of the likelihood. Thus we have $\sup _b |{\hat{Q}}_n(b) - Q_n(b) |=o_p(1)$.

Now let $q_i(x_i,y_i,b)={\mathbf {1}}_{[x_i\in A_x]}(y_{i}\log [F(x_i b)]+(1-y_i)\log [1-F(x_i b)])$. So

$$\begin{aligned} \sup _b |Q_n(b) - Q_0(b)|=\sup _b\left| \frac{1}{n}\sum _{i=1}^n (q_i(x_i,y_i,b)-E[q_i(x_i,y_i,b)])\right| . \end{aligned}$$

(28)

As Ichimura (1993, p. 91), we can use the uniform law of large numbers by Andrews (1987) and so (28) goes to 0 in probability. This completes the proof of theorem 2.