Parameter estimation of regression model with AR(p) error terms based on skew distributions with EM algorithm

Abstract

In the linear regression model, the errors are usually assumed to be uncorrelated. However, in real-life data, this assumption is not often plausible. In this study, first, we will assume that the errors of the regression model have autoregressive structure. This type of regression models has been considered before. However, in those papers under this assumption usually, the symmetric distributions are used as error distribution. The main contribution of this work is to use skew distributions instead of symmetric distributions as error distribution in regression models with autoregressive errors. We provide expectation maximization algorithm to compute the maximum likelihood estimates for the parameters. The performances of the proposed estimators are demonstrated with a simulation study and a real data example. We also provide the confidence intervals using the observed Fisher information matrix for the corresponding estimators.

Appendices

Appendix A: Skew-normal distribution

In “Appendix A” and “Appendix B,” we give the partial second derivatives of the conditional complete likelihood function with respect to the unknown parameters to obtain the observed Fisher information matrix for the regression model defined in Eq. (4) with both skew-normal and skew-t distributed error terms. Note that the observed Fisher information matrices are used to compute the standard errors and the confidence intervals in the simulation study and the real data examples.

$$ \begin{aligned} \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \beta_{k} }} = & - \,\mathop \sum \limits_{t = p + 1}^{N} \left( {\frac{1}{{\sigma^{2} }} + \frac{{\lambda^{3} }}{{\sigma^{2} }} + \frac{{\lambda^{2} }}{{\sigma^{2} }}w_{t}^{2} } \right)\varPhi \left( B \right)x_{t,j} \varPhi \left( B \right)x_{t,k} \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \phi_{i} }} = & - \,\mathop \sum \limits_{t = p + 1}^{N} \left[ {\left( {\frac{1}{{\sigma^{2} }} + \frac{{\lambda^{2} }}{{\sigma^{2} }}w_{t} \left( {\frac{\lambda }{\sigma }\eta_{t} + w_{t} } \right)} \right)e_{t - i} \varPhi \left( B \right)x_{t,j} + \left( {\frac{1}{\sigma }\eta_{t} - \frac{\lambda }{\sigma }w_{t} } \right)x_{t - i,j} } \right] \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \sigma }} = & \mathop \sum \limits_{t = p + 1}^{N} \left( { - \frac{2}{{\sigma^{2} }}\eta_{t} + \frac{\lambda }{{\sigma^{2} }}w_{t} } \right)\varPhi \left( B \right)x_{t,j} - \frac{{\lambda^{2} }}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\frac{\lambda }{\sigma } + w_{t} } \right)w_{t} \eta_{t}^{2} \varPhi \left( B \right)x_{t,j} \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \lambda }} = & \frac{\lambda }{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\frac{1}{\sigma } + \lambda \eta_{t}^{2} + w_{t} \eta_{t} } \right)w_{t} \varPhi \left( B \right)x_{t,j} \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \phi_{r} }} = & - \,\mathop \sum \limits_{t = p + 1}^{N} \left[ {\left( {\frac{1}{{\sigma^{2} }} + \frac{{\lambda^{3} }}{{\sigma^{2} }}w_{t} \eta_{t} + \frac{{\lambda^{2} }}{{\sigma^{2} }}w_{t}^{2} } \right)e_{t - i} e_{t - r} } \right] \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \sigma }} = & \mathop \sum \limits_{t = p + 1}^{N} \left( { - \frac{2}{{\sigma^{2} }}\eta_{t} + \frac{\lambda }{{\sigma^{2} }}w_{t} - \frac{{\lambda^{2} }}{{\sigma^{2} }}\left( {\lambda \eta_{t} + w_{t} } \right)w_{t} \eta_{t} } \right)e_{t - i} \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \lambda }} = & \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\lambda^{2} \eta_{t}^{2} + \lambda w_{t} \eta_{t} - 1} \right)w_{t} e_{t - i} \\ \frac{{\partial^{2} \ln L}}{{\partial \sigma^{2} }} = & \frac{N - p}{{\sigma^{2} }} - \mathop \sum \limits_{t = p + 1}^{N} \left[ {\frac{3}{{\sigma^{2} }}\eta_{t} + \frac{{\lambda^{2} }}{{\sigma^{2} }}\left( {\lambda \eta_{t} + w_{t} } \right)w_{t} \eta_{t} - \frac{2\lambda }{{\sigma^{2} }}w_{t} } \right]\eta_{t} \\ \frac{{\partial^{2} \ln L}}{\partial \sigma \partial \lambda } = & \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\lambda^{2} \eta_{t}^{2} + \lambda w_{t} \eta_{t} - 1} \right)w_{t} \eta_{t} \\ \frac{{\partial^{2} \ln L}}{{\partial \lambda^{2} }} = & - \,\frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\lambda \eta_{t} + w_{t} } \right)w_{t} \eta_{t}^{2} \\ \end{aligned} $$

Here $\eta_{t} = \frac{{\left( {\phi \left( B \right)y_{t} - \phi \left( B \right)x_{t}^{T}\varvec{\beta}} \right)}}{\sigma },$$w_{t} = \frac{{\phi \left( {\lambda \eta_{t} } \right)}}{{\varPhi \left( {\lambda \eta_{t} } \right)}},$$i,r = 1,2, \ldots ,p$ and $j,k = 1,2, \ldots ,M.$

Appendix B: Skew-t distribution

$$ \begin{aligned} \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \beta_{k} }} = & \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\alpha_{t} \eta_{t} - \frac{{w_{t} }}{\sigma } - \lambda w_{t}^{1/2} \left( {\psi_{1,t} w_{t} + \frac{3}{2}\gamma_{t} \alpha_{t} } \right)} \right)\varPhi \left( B \right)x_{t,j} \varPhi \left( B \right)x_{t,k} \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \phi_{i} }} = & \mathop \sum \limits_{t = p + 1}^{N} \left( {\alpha_{t} \eta_{t} - \frac{{w_{t} }}{\sigma } - \lambda w_{t}^{1/2} \left( {\psi_{1,t} w_{t} + \frac{3}{2}\gamma_{t} \alpha_{t} } \right)} \right)e_{t - i} \varPhi \left( B \right)x_{t,j} - \left( {\frac{1}{\sigma }w_{t} \eta_{t} - \lambda \gamma_{t} w_{t}^{3/2} } \right)x_{t - i,j} \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \sigma }} = & \mathop \sum \limits_{t = p + 1}^{N} \left[ { - \frac{1}{{\sigma^{2} }}\left( {w_{t} \eta_{t} - \lambda \gamma_{t} w_{t}^{{\frac{3}{2}}} } \right) + \frac{1}{\sigma }\left( {\alpha_{t} \eta_{t} - \frac{1}{\sigma }w_{t} \eta_{t} - \lambda \left( {\psi_{1,t} w_{t}^{3/2} + \frac{3}{2}\gamma_{t} w_{t}^{1/2} \alpha_{t} } \right)} \right)} \right]\varPhi \left( B \right)x_{t,j} \\ \frac{{\partial^{2} \ln L}}{{\partial \beta_{j} \partial \lambda }} = & - \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\gamma_{t} + \lambda \psi_{2,t} } \right)w_{t}^{3/2} \varPhi \left( B \right)x_{t,j} \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \phi_{r} }} = & \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left[ {\left( {\alpha_{t} \eta_{t} - \frac{{w_{t} }}{\sigma }} \right) - \frac{\lambda }{\sigma }w_{t}^{1/2} \left( {\psi_{1,t} w_{t} + \frac{3}{2}\gamma_{t} \alpha_{t} } \right)e_{t - i} e_{t - r} } \right] \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \sigma }} = & \mathop \sum \limits_{t = p + 1}^{N} \left[ { - \frac{1}{{\sigma^{2} }}\left( {w_{t} \eta_{t} - \frac{\lambda }{\sigma }\gamma_{t} w_{t}^{{\frac{3}{2}}} } \right) + \frac{1}{\sigma }\left( {\alpha_{t} - \frac{{w_{t} }}{\sigma }} \right)\eta_{t} + \frac{\lambda }{{\sigma^{2} }}\gamma_{t} w_{t}^{3/2} } \right]e_{t - i} \\ & \quad - \,\frac{\lambda }{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\psi_{1,t} w_{t}^{3/2} + \frac{3}{2}\gamma_{t} w_{t}^{1/2} \alpha_{t} } \right)e_{t - i} \\ \frac{{\partial^{2} \ln L}}{{\partial \phi_{i} \partial \lambda }} = & - \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\gamma_{t} + \lambda \psi_{2,t} } \right)w_{t}^{3/2} e_{t - i} \\ \frac{{\partial^{2} \ln L}}{{\partial \sigma^{2} }} = & \frac{N - p}{{\sigma^{2} }} - \mathop \sum \limits_{t = p + 1}^{N} \left[ {\frac{3}{{\sigma^{2} }}w_{t} - \frac{1}{\sigma }\alpha_{t} } \right]\eta_{t}^{2} + \mathop \sum \limits_{t = p + 1}^{N} \left[ {\frac{2\lambda }{{\sigma^{2} }}\gamma_{t} w_{t}^{{\frac{3}{2}}} - \frac{\lambda }{\sigma }\left( {\psi_{1,t} w_{t}^{3/2} + \frac{3}{2}\gamma_{t} w_{t}^{1/2} \alpha_{t} } \right)} \right]\eta_{t} \\ \frac{{\partial^{2} \ln L}}{\partial \sigma \partial \lambda } = & - \frac{1}{\sigma }\mathop \sum \limits_{t = p + 1}^{N} \left( {\gamma_{t} + \lambda \psi_{2,t} } \right)w_{t}^{3/2} \eta_{t} \\ \frac{{\partial^{2} \ln L}}{{\partial \lambda^{2} }} = & \frac{\nu + 1}{\nu }\mathop \sum \limits_{t = p + 1}^{N} \psi_{2,t} w_{t}^{1/2} \eta_{t} , \\ \end{aligned} $$

Here $\eta_{t} = \frac{{\left( {\phi \left( B \right)y_{t} - \phi \left( B \right)x_{t}^{T}\varvec{\beta}} \right)}}{\sigma }$, $w_{t} = \frac{\nu + 1}{{\nu + \eta_{t}^{2} }}$, $\gamma_{t} = \frac{\nu }{\nu + 1}\frac{{t_{\nu + 1} \left( {\lambda \eta_{t} \sqrt {w_{t} } } \right) }}{{T_{\nu + 1} \left( {\lambda \eta_{t} \sqrt {w_{t} } } \right)}}$, $i,r = 1,2, \ldots ,p$, $j,k = 1,2, \ldots ,M,$$\alpha_{t} = \frac{2}{\sigma }w_{t} \frac{{\eta_{t} }}{{\nu + \eta_{t}^{2} }}$, $\psi_{1,t} = \frac{{\lambda^{2} }}{\sigma }\frac{{\nu \left( {\nu + 3} \right)}}{{\left( {\nu + 1} \right)}}w_{t}^{2} \eta_{t} \frac{{\gamma_{t} }}{{\left( {\nu + 1 + \lambda^{2} \eta_{t}^{2} w_{t} } \right)}} - \frac{\lambda }{\sigma }\frac{{\left( {\nu + 1} \right)}}{\nu }w_{t}^{3/2} \gamma_{t}^{2},$ and $\psi_{2,t} = - \lambda \left( {\nu + 3} \right)w_{t} \eta_{t}^{2} \frac{{\gamma_{t} }}{{\left( {\nu + 1 + \lambda^{2} \eta_{t}^{2} w_{t} } \right)}} - \frac{{\left( {\nu + 1} \right)}}{\nu }w_{t}^{1/2} \gamma_{t}^{2} \eta_{t}$.