Variable selection for generalized partially linear models with longitudinal data

Abstract

Variables selection and parameter estimation are of great significance in all regression analysis. A variety of approaches have been proposed to tackle this problem. Among those, the penalty-based shrinkage approach has been most popular for the ability to carry out the variable selection and parameter estimation simultaneously. However, not much work is available on the variable selection for the generalized partially models (GPLMs) with longitudinal data. In this paper, we proposed a variable selection procedure for GPLMs with longitudinal data. The inference is based on the SCAD-penalized quadratic inference functions, which is obtained after the B-spline approximating to non-parametric function in the model. The proposed approach efficiently utilized the within-cluster correlation information, which can improve estimating efficiency. The proposed approach also has the virtue of low computational cost. With the tuning parameter chosen by BIC, the correct model is identified with probability tends to 1. The resulted estimator of the parametric component is asymptotic to a normal distribution, and that of the non-parametric function achieves the optimal convergence rate. The performance of the proposed methods is evaluated through extensive simulation studies. A real data analysis shows that the proposed approach succeeds in excluding the insignificant variable.

Appendix: Proofs of the main results

For convenience and simplicity, let $C$ denote a positive constant that may have different values at each appearance throughout this paper and $\parallel A\parallel$ denote the modulus of the largest singular value of matrix or vector A.

Let $\eta_{ij} = X_{ij}^{T} \beta + W_{ij}^{T} \gamma$, Then $\mu_{ij} = h\left( {\eta_{ij} } \right)$. Let $\eta_{i} = (\eta_{i1} , \ldots ,\eta_{im} )^{T} ,\mu_{i} = (\mu_{i1} , \ldots ,\mu_{im} )^{T}$, and $\theta = (\beta^{T} ,\gamma^{T} )^{T} ,Y_{i} = (Y_{i1} , \ldots ,Y_{im} )^{T} ,X_{i} = (X_{i1} , \ldots ,X_{im} )^{T}$.

Similarly, let $P_{ij} = (X_{ij}^{T} ,W_{ij} )^{T}$, and $W_{i} = \left( {W_{i1} , \ldots ,W_{im} } \right))^{T} ,P_{i} = (P_{i1} , \ldots ,P_{im} )^{T} = \left( {X_{i} ,W\left( {U_{i} } \right)} \right)$, then $\eta_{ij} = P_{ij}^{T} \theta ,\eta_{i} = P_{i} \theta$, and $\frac{{\partial \eta_{ij} }}{\partial \theta } = P_{ij} ,\frac{{\partial \eta_{i} }}{\partial \theta } = P_{i}^{T}$.

Let $h^{\prime} \left( t \right) = \frac{dh\left( t \right)}{dt}$, then $\frac{{\partial \mu_{ij} }}{\partial \theta } = h^{\prime} \left( {\eta_{ij} } \right)P_{ij}$. Let

$$H^{\prime} \left( {\eta_{i} } \right) \triangleq \left( {\begin{array}{*{20}l} {h^{\prime} \left( {\eta_{i1} } \right)} \hfill & {} \hfill & {} \hfill \\ {} \hfill & \ddots \hfill & {} \hfill \\ {} \hfill & {} \hfill & {h^{\prime} \left( {\eta_{im} } \right)} \hfill \\ {} \hfill & {} \hfill & {} \hfill \\ \end{array} } \right),\quad H''\left( {\eta_{i} } \right) \triangleq \left( {\begin{array}{*{20}l} {h^{\prime} \prime \left( {\eta_{i1} } \right)} \hfill & {} \hfill & {} \hfill \\ {} \hfill & \ddots \hfill & {} \hfill \\ {} \hfill & {} \hfill & {h^{\prime} \prime \left( {\eta_{i,} } \right)} \hfill \\ {} \hfill & {} \hfill & {} \hfill \\ \end{array} } \right),$$

then

$$\dot{\mu }_{i} = \left( {\begin{array}{*{20}l} {\frac{{\partial \mu_{i1} }}{{\partial \beta_{1} }}} \hfill & \cdots \hfill & {\frac{{\partial \mu_{i1} }}{{\partial \gamma_{qL} }}} \hfill \\ \vdots \hfill & \cdots \hfill & \vdots \hfill \\ {\frac{{\partial \mu_{im} }}{{\partial \beta_{1} }}} \hfill & \cdots \hfill & {\frac{{\partial \mu_{im} }}{{\partial \gamma_{qL} }}} \hfill \\ \end{array} } \right) = \left( {\begin{array}{*{20}l} {\left( {\frac{{\partial \mu_{i1} }}{\partial \theta }} \right)^{T} } \hfill \\ \vdots \hfill \\ {\left( {\frac{{\partial \mu_{im} }}{\partial \theta }} \right)^{T} } \hfill \\ \end{array} } \right) = \left( {\begin{array}{*{20}l} {P_{i1}^{T} h^{\prime} \left( {\eta_{i1} } \right)} \hfill \\ \vdots \hfill \\ {P_{im}^{T} h^{\prime} \left( {\eta_{im} } \right)} \hfill \\ \end{array} } \right) = H^{\prime} \left( {\eta_{i} } \right)P_{i} .$$

Proof of Theorem 1

Let $\delta = n^{{ - \frac{1}{2}}} ,\beta = \beta_{0} + \delta D_{1} ,\gamma = \gamma_{0} + \delta D_{2}$ and $D = (D_{1}^{T} ,D_{2}^{T} )^{T}$. we first show that for any given $\varepsilon > 0$, there exists a large constant C such that

$$P\left\{ {\mathop { \inf }\limits_{\parallel D\parallel = c} {\mathcal{Q}}_{n}^{P} \left( {\beta ,\gamma } \right) > {\mathcal{Q}}_{n}^{P} \left( {\beta_{0} ,\gamma_{0} } \right)} \right\} \ge 1 - \varepsilon .$$

(9.1)

Note that $\beta_{0l} = 0$, for all $l = P_{1} + 1, \cdots ,p$ and $\gamma_{0k} = 0$, for all $k = q_{1} , \cdots ,q$, together with assumption (A1) and $p_{\lambda } \left( 0 \right) = 0$, we have,

$$\begin{aligned} & {\mathcal{Q}}_{n}^{p} \left(\theta \right) - {\mathcal{Q}}_{n}^{p} \left({\theta_{0}} \right) \geqslant \left[{{\mathcal{Q}}_{n} \left(\theta \right) - {\mathcal{Q}}_{n} \left({\theta_{0}} \right)} \right] + n\mathop \sum \limits_{i = 1}^{{p_{1}}} \left[{p_{{\lambda_{2}}} \left({\left| {\beta_{l}} \right|} \right) - p_{{\lambda_{2}}} \left({\left| {\beta_{0l}} \right|} \right)} \right] \\ & \quad + n\mathop \sum \limits_{i = 1}^{{q_{1}}} \left[{p_{{\lambda_{1}}} \left({\parallel \gamma_{k} \parallel_{H}} \right) - p_{{\lambda_{1}}} \left({\parallel \gamma_{0k} \parallel_{H}} \right)} \right] \\ & \quad \triangleq I_{1} + I_{2} + I_{3}. \\ \end{aligned}$$

By Taylor expansion and assumption (A4), we have

$$\begin{aligned} I_{2} & = n\mathop \sum \limits_{l = 1}^{{p_{1}}} \left[{\delta p_{{\lambda_{2}}}^{\prime} \left({\left| {\beta_{0l}} \right|} \right){\text{s}}gn\left({\beta_{0l}} \right)\left| {D_{1l}} \right| + \delta p_{{\lambda_{2}}}^{\prime \prime} \left({\left| {\beta_{0l}} \right|} \right){\text{s}}gn\left({\beta_{0l}} \right)|D_{1l} |^{2} \left\{{1 + o\left(1 \right)} \right\}} \right] \\ & \leqslant \sqrt {p_{1}} a_{n} \parallel D\parallel O\left({n^{1/2}} \right) + b_{n} \parallel D\parallel^{2} O\left(1 \right) = \sqrt {p_{1}} \parallel D\parallel O\left({n^{- 1/2}} \right) + \parallel D\parallel^{2} o\left(1 \right). \\ \end{aligned}$$

Invoking the proof of Theorem 2 in [8],

$${\mathcal{Q}}_{n} \left( \theta \right) - {\mathcal{Q}}_{n} \left( {\theta_{0} } \right) = D^{T} \dot{g}_{N}^{T} \left( {\theta_{0} } \right)\varOmega_{n}^{ - 1} \left( {\theta_{0} } \right)\dot{g}_{N} \left( {\theta_{0} } \right)D + \parallel D\parallel^{2} o_{p} \left( 1 \right) + \parallel D\parallel O_{p} \left( 1 \right).$$

By choosing a sufficient large $C$, $I_{1}$ dominates $I_{2}$. Similarly, $I_{1}$ dominates $I_{3}$ for a sufficient large C. Thus (9.1) holds. i.e., with probability at least $1 - \varepsilon$, there exists a local minimizer $\hat{\theta }$ satisfies that $\parallel \hat{\theta } - \theta_{0} \parallel = O_{p} \left( \delta \right)$. Therefore $\parallel \hat{\gamma } - \gamma_{0} \parallel = O_{p} \left( {n^{ - 1/2} } \right)$ and $\parallel \hat{\beta } - \beta_{0} \parallel = O_{p} \left( {n^{ - 1/2} } \right)$. Follow the proof of Theorem 1 of [8], we have

$$\parallel \hat{\alpha }_{k} \left( u \right) - \alpha_{0k} \left( u \right)\parallel^{2} = O_{p} \left( {n^{{ - \frac{2r}{2r + 1}}} } \right).$$

Thus, we complete the proof of Theorem 1.

Proof of Theorem 2

According to Theorem 1, in order to proof the first part of Theorem 2, we need only to proof that, for any $\gamma$ satisfies $\parallel \gamma - \gamma_{0} \parallel = O_{p} \left( {n^{ - 1/2} } \right)$ and for any $\beta_{l}$ satisfies $\parallel \beta_{l} - \beta_{0l} \parallel = O_{p} \left( {n^{ - 1/2} } \right),l = 1, \ldots ,p_{1}$, there exists a certain $\epsilon = Cn^{- 1/2}$ satisfies that, as $n \to \infty$, with probability tending to 1,

$$\frac{{\partial {\mathcal{Q}}_{n}^{p} \left({\beta,\gamma} \right)}}{{\partial \beta_{l}}} > 0,\quad for\quad 0 < \beta_{l} < epsilon,l = p_{1} + 1, \ldots,p,$$

(9.2)

and

$$\frac{{\partial {\mathcal{Q}}_{n}^{p} \left({\beta,\gamma} \right)}}{{\partial \beta_{l}}} < 0,\quad for\quad - epsilon < \beta_{l} < 0,l = p_{1} + 1, \ldots,p.$$

(9.3)

These imply that the PQIF ${\mathcal{Q}}_{n}^{p} \left( {\beta ,\gamma } \right)$ reaches its minimum at $\beta_{l} = 0,l = p_{1} + 1, \ldots ,p$.

Following Lemma 3 and 4 of [18], we have

$$\begin{aligned} & \frac{{\partial {\mathcal{Q}}_{n}^{p} \left( {\beta ,\gamma } \right)}}{{\partial \beta_{l} }} = \frac{{\partial g_{n}^{T} \left( {\beta ,\gamma } \right)}}{{\partial \beta_{l} }}\varOmega_{n}^{ - 1} \left( {\beta ,\gamma } \right)g_{n} \left( {\beta ,\gamma } \right) + O_{p} \left( 1 \right) + np'_{{\lambda \left( {\left| {\beta_{l} } \right|} \right)}} sgn \left( {\beta_{l} } \right) \\ & \quad = - 2\mathop \sum \limits_{i = 1}^{n} \left( {\begin{array}{*{20}l} {\dot{\mu }_{i}^{T} A_{i}^{ - 1/2} M_{1} A_{i}^{ - 1/2} \frac{{\partial \mu_{i} }}{{\partial \beta_{l} }}} \hfill \\ \vdots \hfill \\ {\dot{\mu }_{i}^{T} A_{i}^{ - 1/2} M_{s} A_{i}^{ - 1/2} \frac{{\partial \mu_{i} }}{{\partial \beta_{l} }}} \hfill \\ \end{array} } \right)^{T} \varOmega_{n}^{ - 1} \left( {\beta ,\gamma } \right)g_{n} \left( {\beta ,\gamma } \right) \\ & \quad + np'_{{\lambda \left( {\left| {\beta_{l} } \right|} \right)}} sgn \left( {\beta_{l} } \right) + O_{p} \left( 1 \right) \\ & \quad = n^{1/2} \left[ {n^{1/2} \lambda \left\{ {\lambda^{ - 1} p'_{\lambda } \left( {\left| {\beta_{l} } \right|} \right) sgn \left( {\beta_{l} } \right)} \right\} + O_{p} \left( 1 \right)} \right]. \\ \end{aligned}$$

(9.4)

According to (3.7), the expression of the derivative of SCAD penalized function, it easy to see that ${ \lim }_{n \to \infty } { \liminf }_{{\beta_{l} \to 0}} \lambda^{ - 1} p'_{\lambda } \left( {\left| {\beta_{l} } \right|} \right) = 1$. Together with the Assumption (A10), $\lambda n^{1/2} > \lambda_{ \hbox{min} } n^{1/2} \to \infty$, it is clear that the sign of (9.4) is decided by that of $\beta_{l}$. This implies (9.2) and (9.3) hold. Thus we complete the proof of Theorem 2 .

Proof of Theorem 3

Let $\theta^{ *} = (\beta^{ *T} ,\gamma^{ *T} )^{T}$, and let $P_{i}^{ *} = (X_{i}^{ *T} ,W_{i}^{ *T} )^{T} , i = 1, \ldots ,n$ denote the covariates corresponding to $\theta^{ *}$. Denote ${\dot{\mathcal{Q}}}_{1n} \left( {\beta ,\gamma } \right)$ and ${\dot{\mathcal{Q}}}_{2n} \left( {\beta ,\gamma } \right)$ to be the first derivatives of the PQIF ${\mathcal{Q}}_{n}^{p}$ with respect to $\beta$ and $\gamma$ respectively. i.e.,

$${\dot{\mathcal{Q}}}_{1n} \left( {\beta ,\gamma } \right) = \frac{{\partial {\mathcal{Q}}_{n}^{p} \left( {\beta ,\gamma } \right)}}{\partial \beta },{\dot{\mathcal{Q}}}_{2n} \left( {\beta ,\gamma } \right) = \frac{{\partial {\mathcal{Q}}_{n}^{p} \left( {\beta ,\gamma } \right)}}{\partial \gamma }.$$

By Theorems 1 and 2, $(\hat{\beta }^{ *T} ,0^{T} )^{T}$ and $\hat{\gamma }^{ *T}$ satisfies that

$${\dot{\mathcal{Q}}}_{1n} (((\hat{\beta }^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\hat{\gamma }^{ *T} )^{T} ) = {\mathbf{0}}^{T} \,and\,{\dot{\mathcal{Q}}}_{2n} (((\hat{\beta }^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\hat{\gamma }^{ *T} )^{T} ) = {\mathbf{0}}^{T}$$

By Taylor expansion, we have

$$\begin{aligned} & {\mathcal{Q}}_{1n} |_{{((\hat{\beta }^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\hat{\gamma }^{ *T} )^{T} }} = {\mathcal{Q}}_{1n} |_{{((\beta_{0}^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\gamma_{0}^{ *T} )^{T} }} + \frac{{\partial {\mathcal{Q}}_{1n} }}{\partial \beta }|_{{\theta = \tilde{\theta }}} \left\{ {(\hat{\beta }^{ *T} ,{\mathbf{0}}^{T} )^{T} - (\beta_{0}^{ *T} ,{\mathbf{0}}^{T} )^{T} } \right\} \\ & \quad + \frac{{\partial {\mathcal{Q}}_{1n} }}{\partial \gamma }|_{{\theta = \tilde{\theta }}} \left\{ {\hat{\gamma }^{ *} - \hat{\gamma }_{0}^{ *} } \right\} + \mathop \sum \limits_{i = 1}^{{p_{1} }} np_{\lambda }^{\prime } \left( {\left| {\hat{\beta }_{l} } \right|} \right){\text{s}}gn\left( {\hat{\beta }_{l} } \right), \\ \end{aligned}$$

where $\tilde{\theta }$ is between $((\beta_{0}^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\gamma_{0}^{ *T} )^{T}$ and $((\hat{\beta }^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\hat{\gamma }^{ *T} )^{T}$. Apply Taylor expansion to $p_{\lambda }^{\prime } \left( {\left| {\hat{\beta }_{l} } \right|} \right)$, we obtain

$$p_{\lambda }^{\prime } \left( {\left| {\hat{\beta }_{l} } \right|} \right) = p_{\lambda }^{\prime } \left( {\left| {\beta_{0l} } \right|} \right) + \{ p_{{\lambda_{2l} }}^{\prime \prime } (|\beta_{0l} | + o_{p} (1)\} \left( {\hat{\beta }_{l} - \beta_{0l} } \right).$$

by assumption (A10), $p''_{\lambda } \left( {\left| {\beta_{0l} } \right|} \right) = o_{p} \left( 1 \right)$. Note $p'_{\lambda } \left( {\left| {\beta_{0l} } \right|} \right) = 0$ as $\lambda_{ \hbox{max} } \to 0$, therefore, by Lemma 4 of [18] and through some calculation, we have

$$\begin{aligned} & \frac{1}{n}{\mathcal{Q}}_{1n} |_{{((\beta_{0}^{ *T} ,{\mathbf{0}}^{T} )^{T} ,\gamma_{0}^{ *T} )^{T} }} = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \mathop \sum \limits_{k = 1}^{s} \mathop \sum \limits_{l = 1}^{s} \left\{ {X_{i}^{ *T} H^{\prime} \left( {\eta_{i} } \right)A_{i}^{ - 1/2} M_{k} A_{i}^{ - 1/2} H^{\prime} \left( {\eta_{i} } \right)P_{i}^{ *} } \right. \\ & \quad \cdot \left. {\varOmega_{kl}^{ - 1} P_{j}^{ *T} H^{\prime} \left( {\eta_{j} } \right)A_{j}^{ - 1/2} M_{l} A_{j}^{ - 1/2} \left( {Y_{j} - \mu_{0j} } \right)} \right\} + o_{p} \left( {n^{ - 1/2} } \right) \\ & \quad = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} X_{i}^{ *T} H^{\prime} \left( {\eta_{i} } \right)\tau_{ij} \left( {Y_{j} - \mu_{0j} } \right) + o_{p} \left( {n^{ - 1/2} } \right) \\ = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \tilde{X}_{i}^{T} \tau_{ij} \left( {\tilde{R}\left( {U_{j} } \right) + \varepsilon_{j} } \right) + o_{p} \left( {n^{ - 1/2} } \right), \\ \end{aligned}$$

where $\tilde{X}_{i} = H^{\prime} \left( {\eta_{i} } \right)X_{i}^{ *} ,\tilde{R}\left( {U_{i} } \right) = H^{\prime} \left( {\eta_{i} } \right)R\left( {U_{i} } \right),\varOmega_{kl}^{ - 1}$ is the $\left( {l,k} \right)$ block of $\varOmega^{ - 1}$ and

$$\tau_{ij} = \mathop \sum \limits_{k = 1}^{s} \mathop \sum \limits_{l = 1}^{s} A_{i}^{ - 1/2} M_{k} A_{i}^{ - 1/2} H^{\prime} \left( {\eta_{i} } \right)P_{i}^{ *} \varOmega_{kl}^{ - 1} P_{j}^{ *T} H^{\prime} \left( {\eta_{j} } \right)A_{j}^{ - 1/2} M_{l} A_{j}^{ - 1/2} .$$

Similarly, we have

$$\begin{aligned} & \frac{1}{n}\frac{{\partial {\mathcal{Q}}_{1n} }}{\partial \beta }|_{{\theta = \tilde{\theta }}} = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \tilde{X}_{i}^{T} \tau_{ij} \tilde{X}_{j} + o_{p} \left( {n^{ - 1/2} } \right), \\ & \quad \frac{1}{n}\frac{{\partial {\mathcal{Q}}_{1n} }}{\partial \gamma }|_{{\theta = \tilde{\theta }}} = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \tilde{X}_{i}^{T} \tau_{ij} \tilde{W}\left( {U_{j} } \right) + o_{p} \left( {n^{ - 1/2} } \right), \\ \end{aligned}$$

where, $\tilde{W}\left( {U_{j} } \right) = H^{\prime} \left( {\eta_{j} } \right)W\left( {U_{j} } \right),W\left( {U_{j} } \right) = (W_{j1} , \ldots ,W_{jm} )^{T} ,W_{ij} = B\left( {U_{ij} } \right)$. Hence

$$\begin{aligned} & \frac{1}{n}{\mathcal{Q}}_{1n} |_{{((\beta_{0}^{ *T} ,{\mathbf{0}}^{T} 0)^{T} ,\gamma_{0}^{ *T} )^{T} }} = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \tilde{X}_{i}^{T} \tau_{ij} \left\{ {\tilde{X}_{j} \left( {\beta_{0}^{ *} - \hat{\beta }^{ *} } \right)} \right. \\ & \quad \left. { +\; \tilde{W}\left( {U_{j} } \right) \cdot \left( {\gamma_{0}^{ *} - \hat{\gamma }^{ *} } \right) + \tilde{R}\left( {U_{j} } \right) + \varepsilon_{j} } \right\} + o_{p} \left( {\hat{\beta }^{ *} - \beta_{0}^{ *} } \right), \\ & \quad \frac{1}{n}{\mathcal{Q}}_{2n} |_{{((\beta_{0}^{ *T} ,{\mathbf{0}}^{T} 0)^{T} ,\gamma_{0}^{ *T} )^{T} }} = - \frac{2}{{n^{2} }}\mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{j = 1}^{n} \tilde{W}(U_{i} )^{T} \tau_{ij} \left\{ {\tilde{X}_{j} \left( {\beta_{0}^{ *} - \hat{\beta }^{ *} } \right)} \right. \\ & \quad \left. { +\; \tilde{W}\left( {U_{j} } \right) \cdot \left( {\gamma_{0}^{ *} - \hat{\gamma }^{ *} } \right) + \tilde{R}\left( {U_{j} } \right) + \varepsilon_{j} } \right\} + o_{p} \left( {\hat{\gamma }^{ *} - \gamma_{0}^{ *} } \right). \\ \end{aligned}$$

Follow the proof of Theorem 2 in [18], we prove (4.3). Thus we complete the proof of Theorem 3.