Robust exponential squared loss-based estimation in semi-functional linear regression models

Abstract

In this paper, we present a new robust estimation procedure for semi-functional linear regression models by using exponential squared loss. The outstanding advantage of the proposed method is the resulting estimators are more efficient than the least squares estimators in the presence of outliers or heavy-tail error distributions. The slope function and functional predictor variable are approximated by functional principal component basis functions. Under some regularity conditions, we obtain the optimal convergence rate of slope function, and the asymptotic normality of parameter vector and variance estimator. Finally, we investigate the finite sample performance of the proposed method through a simulation study and real data analysis.

Appendix A: Technical proofs

Proof of Theorem 1

Let $\delta _n=n^{-\frac{2b-1}{2(a+2b)}}$, $\varvec{\alpha }=\varvec{\alpha }_0+\delta _n\varvec{u}_1$, $\varvec{\gamma }=\varvec{\gamma }_0+\delta _n\varvec{u}_2$, $\varvec{u}=\big (\varvec{u}_1^T,\varvec{u}_2^T\big )^T$, $R_i=\int _{0}^{1}\beta _0(t)X_i(t)dt-\varvec{U}_i^T\varvec{\gamma }_0$. We next show that, for any given $\eta >0$, there exists a sufficient large constant L such that

$$\begin{aligned} {\lim \inf }_{n} P\left\{ \sup _{\Vert \varvec{u}\Vert =L}Q_n(\varvec{\alpha }, \varvec{\gamma })<Q_n(\varvec{\alpha }_0, \varvec{\gamma }_0)\right\} \ge 1-\eta . \end{aligned}$$

(A.1)

This implies with the probability at least $1-\eta$ that there exists a local maximizer $\varvec{\hat{\alpha }}$ and $\varvec{\hat{\gamma }}$ in the ball $\Big \{(\varvec{\alpha }_0^T,\varvec{\gamma }_0^T)^T+\delta _n\varvec{u}:\big \Vert \varvec{u}\big \Vert \le L\Big \}$ such that $\Vert \varvec{\hat{\alpha }}-\varvec{\alpha }_0\Vert =O_p(\delta _n)$ and $\Vert \varvec{\hat{\gamma }}-\varvec{\gamma }_0\Vert =O_p(\delta _n)$, which is exactly what we want to show.

Note that $\Vert v_j-\hat{v}_j\Vert ^2=O_p(n^{-1}j^2)$ (see, e.g., Shin 2009; Yu et al. 2016), one has

$$\begin{aligned} \begin{aligned} |R_i|^2&=\Big |\int _{0}^{1}\beta _0(t)X_i(t)dt-\varvec{U}_i^T\varvec{\gamma }_0\Big |^2\\&\le 2 \Big |\sum _{j=1}^{m}\langle X_i,\hat{v}_j-{v}_j\rangle \gamma _{0j}\Big |^2+2\Big |\sum _{j=m+1}^{ \infty }\langle X_i,v_j\rangle \gamma _{0j}\Big |^2\\&\triangleq 2\text {A}_1+2\text {A}_2. \end{aligned} \end{aligned}$$

For $\text {A}_1$, by condition C1 and the Hölder inequality, it is obtained

$$\begin{aligned} \begin{aligned} \text {A}_1&=\Big |\sum _{j=1}^{m}\langle X_i,{v}_j-\hat{v}_j\rangle \gamma _{0j}\Big |^2 \le cm\sum _{j=1}^{m}\Vert {v}_j-\hat{v}_j\Vert ^2|\gamma _{0j}|^2\\&\le cm\sum _{j=1}^{m}O_p\left( n^{-1}j^{2-2b}\right) =O_p\left( n^{-\frac{a+4b-4}{a+2b}}\right) =o_p\left( \delta _n^2\right) . \end{aligned} \end{aligned}$$

As for $\text {A}_2$, due to

$$\begin{aligned} \begin{aligned}&E\left\{ \sum _{j=m+1}^{ \infty }\langle X_i,v_j\rangle \gamma _{0j}\right\} =0,\\&\text {Var}\left\{ \sum _{j=m+1}^{ \infty }\langle X_i,{v}_j\rangle \gamma _{0j}\right\} =\sum _{j=m+1}^{ \infty }\lambda _j{\gamma _{0j}}^2 \le C \sum _{j=m+1}^{ \infty }j^{-(a+2b)} =O\left( n^{-\frac{a+2b-1}{a+2b}}\right) , \end{aligned} \end{aligned}$$

one has

$$\begin{aligned} A_2=O_p\left( n^{-\frac{a+2b-1}{a+2b}}\right) =o_p\left( \delta _n^2\right) . \end{aligned}$$

Taking these together, we have

$$\begin{aligned} |R_i|^2=o_p\left( \delta _n^2\right) . \end{aligned}$$

(A.2)

Invoking Taylor expansion and a simple calculation, we have

$$\begin{aligned} \begin{aligned} Q_n(\varvec{\alpha }, \varvec{\gamma })-Q(\varvec{\alpha }_0, \varvec{\gamma }_0) =\,&\delta _n\sum _{i=1}^{n}\varphi '_h(\epsilon _i+R_i)\left( \varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2\right) \\&\quad +\frac{\delta _n^2}{2}\varphi ''_h(\epsilon _i+R_i)\left( \varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2\right) ^2\\&\quad +\frac{\delta _n^3}{6}\varphi '''_h(\epsilon '_i+R_i)\left( \varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2\right) ^3\\&\triangleq B_1+B_2+B_3, \end{aligned} \end{aligned}$$

(A.3)

where $\epsilon '_i$ is between $\epsilon _i+R_i$ and $\epsilon _i+R_i-\delta _n(\varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2)$. Furthermore, we have

$$\begin{aligned} \begin{aligned}&\sum _{i=1}^{n}\varphi '_h(\epsilon _i+R_i)\Big (\varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2\Big )\\&\qquad =\sum _{i=1}^{n}\Big \{\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)R_i+\frac{1}{2}\varphi '''_h(\tilde{\epsilon }_i)R_i^2\Big \}\Big (\varvec{z}_i^T\varvec{u}_1+\varvec{U}_i^T\varvec{u}_2\Big ), \end{aligned} \end{aligned}$$

(A.4)

where $\tilde{\epsilon }_i$ is between $\epsilon _i$ and $\epsilon _i+R_i$. Then, by condition C1, Eq. (A.4) and a simple calculation, we can prove that

$$\begin{aligned} B_1=O_p(n\delta _n\Vert \varvec{u}\Vert ). \end{aligned}$$

(A.5)

Similarly, we can obtain

$$\begin{aligned} B_2=O_p(n\delta _n^2\Vert \varvec{u}\Vert ^2), \quad B_3=O_p(n\delta _n^3\Vert \varvec{u}\Vert ^3). \end{aligned}$$

(A.6)

Therefore, by choosing a sufficiently large L, both $B_1$ and $B_3$ are dominated by $B_2$ uniformly in $\Vert \varvec{u}\Vert =L$. Hence, Eq. (A.1) holds with probability tending to one, and there exists local maximizer $\hat{\varvec{\gamma }}$ such that

$$\begin{aligned} \Vert \varvec{\hat{\gamma }}-\varvec{\gamma }_0\Vert =O_p(\delta _n). \end{aligned}$$

(A.7)

Observe that

$$\begin{aligned}\begin{aligned} \Vert \hat{\beta }(t)-\beta _0(t)\Vert ^2&=\bigg \Vert \sum _{j=1}^{m}\hat{\gamma }_{j}\hat{v}_j-\sum _{j=1}^{\infty }{\gamma }_{0j}{v}_j\bigg \Vert ^2\\&\le 2\bigg \Vert \sum _{j=1}^{m}\hat{\gamma }_{j}\hat{v}_j-\sum _{j=1}^{m}{\gamma }_{0j}{v}_j\bigg \Vert ^2+ 2\bigg \Vert \sum _{j=m+1}^{\infty }{\gamma }_{0j}{v}_j\bigg \Vert ^2\\&\le 4\bigg \Vert \sum _{j=1}^{m}(\hat{\gamma }_j-\gamma _{0j})\hat{v}_j\bigg \Vert ^2+ 4\bigg \Vert \sum _{j=1}^{m}{\gamma }_{0j}(\hat{v}_j-v_j)\bigg \Vert ^2+2\sum _{j=m+1}^{\infty }{\gamma }_{0j}^2\\&\triangleq 4D_1+4D_2+2D_3. \end{aligned}\end{aligned}$$

Invoking Eq. (A.7), condition C2, the orthogonality of $\{\hat{v}_j\}$ and $\Vert v_j-\hat{v}_j\Vert ^2=O_p(n^{-1}j^2)$, one has

$$\begin{aligned} D_1= & {} \bigg \Vert \sum _{j=1}^{m}(\hat{\gamma }_j-\gamma _{0j})\hat{v}_j\bigg \Vert ^2 \le \sum _{j=1}^{m}\big |\hat{\gamma }_j-\gamma _{0j}\big |^2\nonumber \\= & {} \Vert \hat{\varvec{\gamma }}-\varvec{\gamma }_0\Vert ^2= O_p(\delta _n^2).\end{aligned}$$

(A.8)

$$\begin{aligned} D_2= & {} \bigg \Vert \sum _{j=1}^{m}{\gamma }_{0j}(\hat{v}_j-v_j)\bigg \Vert ^2\le m\sum _{j=1}^{m}\parallel \hat{v}_j-v_j\parallel ^2{\gamma }_{0j}^2\le \frac{m}{n}O_p\big (\sum _{j=1}^{m}j^2{\gamma }_{0j}^2\big )\nonumber \\= & {} O_p\Big (n^{-1}m\sum _{j=1}^{m}j^{2-2b}\Big )=O_p(n^{-1}m)=o_p(n^{-\frac{2b-1}{a+2b}})=o_p(\delta _n^2). \end{aligned}$$

(A.9)

$$\begin{aligned} D_3= & {} \sum _{j=m+1}^{\infty }\gamma _{0j}^2\le C\sum _{j=m+1}^{\infty }j^{-2b}=O(n^{-\frac{2b-1}{a+2b}})=O(\delta _n^2). \end{aligned}$$

(A.10)

Then, combining Eqs. (A.8)–(A.10) yields Theorem 1. $\square$

Proof of Theorem 2

According to Theorem 1, we know that, as $n\rightarrow \infty$, with probability tending to 1, $Q_n(\varvec{\alpha }, \varvec{\gamma })$ attains the maximal value at $(\hat{\varvec{\alpha }}^T,\hat{\varvec{\gamma }}^T)^T$. Then, we have

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\varvec{z}_i \varphi '_h\big (Y_i-\varvec{z}_i^T \varvec{\hat{\alpha }}-\varvec{U}_i^T\varvec{\hat{\gamma }}\big )=0, \end{aligned}$$

(A.11)

and

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\varvec{U}_i \varphi '_h\big (Y_i-\varvec{z}_i^T \varvec{\hat{\alpha }}-\varvec{U}_i^T\varvec{\hat{\gamma }}\big )=0. \end{aligned}$$

(A.12)

Invoking Taylor expansion and Eq. (A.11), some simple calculation yields

$$\begin{aligned} \begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}\varvec{z}_i \bigg \{\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)\big [R_i-\varvec{z}_i^T (\varvec{\hat{\alpha }}-\varvec{\alpha }_0)-\varvec{U}_i^T(\varvec{\hat{\gamma }}-\varvec{\gamma }_0)\big ]\\&\qquad \qquad \qquad +\frac{1}{2}\varphi '''_h(\epsilon _i^*)\big [R_i-\varvec{z}_i^T (\varvec{\hat{\alpha }}-\varvec{\alpha }_0)-\varvec{U}_i^T(\varvec{\hat{\gamma }}-\varvec{\gamma }_0)\big ]^2\bigg \}=0. \end{aligned}\end{aligned}$$

(A.13)

where $\epsilon _i^*$ is between $\epsilon _i$ and $Y_i-\varvec{z}_i^T \varvec{\hat{\alpha }}-\varvec{U}_i^T\varvec{\hat{\gamma }}$. Similarly, by Eq. (A.12) we can get

$$\begin{aligned} \begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}\varvec{U}_i \bigg \{\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)\big [R_i-\varvec{z}_i^T (\varvec{\hat{\alpha }}-\varvec{\alpha }_0)-\varvec{U}_i^T(\varvec{\hat{\gamma }}-\varvec{\gamma }_0)\big ]\\&\qquad \qquad \qquad +\frac{1}{2}\varphi '''_h(\epsilon _i^{**})\big [R_i-\varvec{z}_i^T (\varvec{\hat{\alpha }}-\varvec{\alpha }_0)-\varvec{U}_i^T(\varvec{\hat{\gamma }}-\varvec{\gamma }_0)\big ]^2\bigg \}=0. \end{aligned}\end{aligned}$$

(A.14)

where $\epsilon _i^{**}$ is between $\epsilon _i$ and $Y_i-\varvec{z}_i^T \varvec{\hat{\alpha }}-\varvec{U}_i^T\varvec{\hat{\gamma }}$.

Let $\Phi _n=\frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i)\varvec{U}_i\varvec{U}^T_i$, $\Psi _n=\frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i)\varvec{U}_i\varvec{z}_i^T$, $\Lambda _n=\frac{1}{n}\sum _{i=1}^{n}\varvec{U}_i(\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)R_i)$. Then, by Eq. (A.14), we have

$$\begin{aligned} \hat{\varvec{\gamma }}-\varvec{\gamma }_0=\big (\Phi _n+o_p(1)\big )^{-1}\big [\Lambda _n+\Psi _n(\varvec{\gamma }_0-\varvec{\hat{\gamma }}_0)\big ]. \end{aligned}$$

(A.15)

Substituting Eq. (A.15) into Eq. (A.13), we can obtain that

$$\begin{aligned} \begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i)\varvec{z}_i[\varvec{z}_i-\Psi _n^T\Phi _n^{-1}\varvec{U}_i]^T(\varvec{\hat{\alpha }}-\varvec{\alpha }_0)+o_p(\varvec{\hat{\alpha }}-\varvec{\alpha }_0)\\&\quad =\frac{1}{n}\sum _{i=1}^{n}\varvec{z}_i\Big (\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)R_i-\varphi ''_h(\epsilon _i)\varvec{U}_i^T\Phi _n^{-1}\Lambda _n\Big ). \end{aligned}\end{aligned}$$

(A.16)

Note that

$$\begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i){\Psi }_n^T\Phi _n^{-1}\varvec{U}_i[\varvec{z}_i-\Psi _n^T\Phi _n^{-1}\varvec{U}_i]^T=0,\qquad \quad \end{aligned}$$

(A.17)

$$\begin{aligned}&\frac{1}{n}\sum _{i=1}^{n}{\Psi }_n^T\Phi _n^{-1}\varvec{U}_i\big [\varphi '_h(\epsilon _i)+\varphi ''_h(\epsilon _i)R_{i}-\varvec{U}_i^\tau \Phi _n^{-1}\Lambda _n\big ]=0. \end{aligned}$$

(A.18)

Then, by Eqs. (A.16)–(A.18), it is easy to show that

$$\begin{aligned} \left( \frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i)\widetilde{\varvec{z}}_i\widetilde{\varvec{z}}_i^T\right) \sqrt{n} (\hat{\varvec{\alpha }}-{\varvec{\alpha }}_0) =\frac{1}{\sqrt{n}}\sum _{i=1}^{n}\varphi '_h(\epsilon _i)\widetilde{\varvec{z}}_i+o_p(1), \end{aligned}$$

(A.19)

where $\widetilde{\varvec{z}}_i={\varvec{z}}_i-{\Psi }_n^T\Phi _n^{-1}\varvec{U}_i$. Furthermore, invoking Lemma 1 in Yu et al. (2016) and condition C6, a simple calculation yields

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\widetilde{\varvec{z}}_i\widetilde{\varvec{z}}_i^T=\hat{\varvec{\Sigma }}\overset{p}{\longrightarrow }\varvec{\Sigma }. \end{aligned}$$

(A.20)

Using the central limits Theorem, we have

$$\begin{aligned} \frac{1}{\sqrt{n}}\sum _{i=1}^{n}\varphi '_h(\epsilon _i)\widetilde{\varvec{z}}_i\overset{d}{\longrightarrow }N\big (0,g(h)\varvec{\Sigma }\big ). \end{aligned}$$

(A.21)

Moreover, by the law of large numbers, we can obtain

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}\varphi ''_h(\epsilon _i)\widetilde{\varvec{z}}_i\widetilde{\varvec{z}}_i^T\overset{p}{\longrightarrow }H(h)\varvec{\Sigma }. \end{aligned}$$

(A.22)

Then, invoking Eqs. (A.19)–(A.22), and the Slutsky’s theorem, we can complete the proof of Theorem 2. $\square$

Proof of Theorem 3

The proof can be obtained by used an argument analogous to Theorem 3 in Zhou et al. (2016), we omit the detail for saving space. $\square$

Proof of Corollary 2

By Theorem 2, we have

$$\begin{aligned} \sqrt{n}\varvec{\Xi }^{-\frac{1}{2}}(\hat{\varvec{\alpha }}-\varvec{\alpha }_0)\overset{d}{\longrightarrow }N\big (0,\varvec{I}_p). \end{aligned}$$

(23)

Note that

$$\begin{aligned} \epsilon _i-\hat{\epsilon }_i=\varvec{z}_i^T(\hat{\varvec{\alpha }}-\varvec{\alpha })+\varvec{U}_i^T(\hat{\varvec{\gamma }}-\varvec{\gamma }_0)-R_i. \end{aligned}$$

By Theorem 1,Theorem 2 and Eq. (A.2), we can get

$$\begin{aligned} \epsilon _i-\hat{\epsilon }_i=O_p(n^{-{\frac{2b-1}{2(a+2b)}}})=o_p(1). \end{aligned}$$

Moreover, according to Lemma 1 in Yu et al. (2016), we have

$$\begin{aligned} \hat{\varvec{\Sigma }}\overset{p}{\longrightarrow }\varvec{\Sigma }. \end{aligned}$$

Thus, invoking the Slutsky’s theorem and the definition of $\varvec{\Xi }$ we complete the proof of Corollary 2. $\square$