Abstract
In their recent paper, Wang and Leblanc (Ann Inst Stat Math 60:883–900, 2008) have shown that the second-order least squares estimator (SLSE) is more efficient than the ordinary least squares estimator (OLSE) when the errors are independent and identically distributed with non zero third moments. In this paper, we generalize the theory of SLSE to regression models with autocorrelated errors. Under certain regularity conditions, we establish the consistency and asymptotic normality of the proposed estimator and provide a simulation study to compare its performance with the corresponding OLSE and generalized least square estimator (GLSE). It is shown that the SLSE performs well giving relatively small standard error and bias (or the mean square error) in estimating parameters of such regression models with autocorrelated errors. Based on our study, we conjecture that for less correlated data, the standard errors of SLSE lie between those of the OLSE and GLSE which can be interpreted as adding the second moment information can improve the performance of an estimator.
Similar content being viewed by others
References
Abarin T (2008) Second-order least squares estimation in regression models with application to measurement error problems. Dissertation, Department of Statistics University of Manitoba, Winnipeg
Abarin T, Wang L (2006) Comparison of GMM with second-order least squares estimator in nonlinear models. Far East J Theor Stat 20:179–196
Abarin T, Wang L (2009) Second-order least squares estimation of censored regression models. J Stat Plan Inference 139:125–135
Amemiya T (1973) Regression analysis when the dependent variable is truncated normal. Econometrica 41:997–1016
Chen X, Tsao M, Zhou J (2011) Robust second-order least-squares estimator for regression models. Stat Pap. doi:10.1007/s00362-010-0343-4
Chipman J (1979) Efficiency of least squares estimation of linear trend when residuals are autocorrelated. Econometrica 47:115–128
Gallant AR, Goebel JJ (1976) Nonlinear regression with autocorrelated errors. J Am Stat Assoc 71(356):961–967
Kim M, Ma Y (2011) The efficiency of the second-order nonlinear least squares estimator and its extension. Ann Inst Stat Math. doi:10.1007/s10463-011-0332-y
Krämer W (1980) Finite sample efficiency of ordinary least squares in the linear regression model with autocorrelated errors. J Am Stat Assoc 75:1005–1009
Krämer W, Marmol F (2002) OLS-based asymptotic inference in linear regression models with trending regressors and AR(p)-disturbances. Commun Stat Theory Methods 31:261–270
Li H (2011) Second-order least squares estimation in generalized linear mixed models. Dissertation, Department of Statistics, University of Manitoba, Winnipeg
Li H, Wang L (2011) A consistent simulation-based estimator in generalized linear mixed models. J Stat Comput Simul. doi:10.1080/00949655.2011.569721
Prais SJ, Winsten CB (1954) Trend estimators and serial correlation. Cowless Commission discussion paper, Statistics no 383
Ullah A, Srivastava VK, Magee L, Srivastava A (1983) Estimation of linear regression model with autocorrelated disturbances. J Time Ser Anal 4(2):127–135
Wang L (2003) Estimation of nonlinear Berkson-type measurement error models. Stat Sin 13:1201–1210
Wang L (2004) Estimation of nonlinear models with Berkson measurement errors. Ann Stat 32:2559–2579
Wang L, Leblanc A (2008) Second-order nonlinear least squares estimation. Ann Inst Stat Math 60:883–900
White H (1980) Nonlinear regression on cross-section data. Econometrica 48:721–746
White H, Domowitz I (1984) Nonlinear regression with dependent observations. Econometrica 52:143–161
Acknowledgments
The financial support from DIKTI Indonesia via Program Academic Recharging (PAR) had greatly helped D. Rosadi to initiate this project in 2011. The financial support from Hibah Kompetensi in 2012 and 2013 is also gratefully acknowledged. This work has been completed while D. Rosadi was visiting the School of Mathematics and Statistics, The University of Sydney in 2011 and 2012. The authors would like to thank the anonymous referee and the editor of this journal for their constructive comments and useful suggestions to improve the quality and readability of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendices
1.1 Appendix 1: Proof of Theorem 1
We will show that assumption of the theorem will fulfill the condition of Lemma 3 of Amemiya (1973). Assumption 1 implies that \({Q_N}(\psi )\) is measurable and continuous in \(\psi \in \varGamma \) with probability one. Using Assumption 2 and 3 and Cauchy Schwarz inequality we have
and
which imply \(\mathop {\sup }\nolimits _\varGamma \left\| \rho _i^{'}(\psi ){W_i}{\rho _i}(\psi ) \right\| \!\!\le \!\! \left\| {{W_i}} \right\| \mathop {\sup }\nolimits _\varGamma {\left\| {{\rho _i}{{(\psi )}}} \right\| }\) where \(E\left\| {{W_i}} \right\| ^ {1+\delta }\mathop {\sup }\nolimits _\varGamma {\left\| {{\rho _i}{{(\psi )}}} \right\| ^{2+\delta }} < \infty \) for some \(\delta >0\).
It follows from the uniform strong law of large numbers of White and Domowitz (1984), Theorem 2.3. that \({\textstyle {1 \over N}}{Q_N}(\psi )\) and \( Q(\psi ) = \frac{1}{N}\sum \nolimits _{i = 1}^N {E(\rho _i^{'}(\psi ){W_i}{\rho _i}(\psi ))} \) converge almost surely and uniformly for all \(\psi \) in \(\varGamma \) to the same limit, let say \( {\bar{Q}}(\psi )\). Furthermore, \({({\rho _i}(\psi ) - {\rho _i}({\psi _0}))}\) does not depend on \(Y_i\), we have
which implies \(Q(\psi ) = Q({\psi _0}) + \frac{1}{N}\sum \nolimits _{i = 1}^N {E\left[ {({\rho _i}(\psi ) \!-\! {\rho _i}({\psi _0})){'}{W_i}({\rho _i}(\psi ) \!-\! {\rho _i}({\psi _0}))} \right] }\). It follows that \(Q(\psi ) \ge Q({\psi _0}) \) and by Assumption 4, equality holds if and only if \(\psi = \psi _0\). Thus by applying Lemma 3 of Amemiya (1973), we have \({{\hat{\psi }}_{SLSE}}\xrightarrow {{a.s}}{\psi _0}\), as \(N \rightarrow \infty \).
1.2 Appendix 2: Proof of Theorem 2
The first derivative \(\partial Q_{N} (\psi )/\partial (\psi )\) exists by Assumption 5 and it has the first-order Taylor expansion in \(\varGamma \). Since \({{\hat{\psi }}_{SLSE}}\xrightarrow {{a.s}}{\psi _0}\) for sufficiently large \(N\) we have
where \(\left\| {{{\tilde{\psi }}_N} - {\psi _0}} \right\| \leqslant \left\| {{{\hat{\psi }}_{SLSE}} - {\psi _0}} \right\| \). The first derivative of \(Q_{N}(\psi )\) in Eq. (5) is given by
where
The second derivative of \(Q_N(\psi )\) in Eq. (5) is given by
where
Here we obtain \( \mathop {\sup }\nolimits _\varGamma \left\| {\frac{{\partial \rho _i^{'}(\psi )}}{{\partial \psi }}{W_i}\frac{{\partial {\rho _i}(\psi )}}{{\partial \psi '}}} \right\| \leqslant \left\| {{W_i}} \right\| \mathop {\sup }\nolimits _\varGamma {\left\| {\frac{{\partial \rho _i^{'}(\psi )}}{{\partial \psi }}} \right\| ^2}\). Using Assumption 5, Cauchy Schwarz inequality, Eq. (6) and similar arguments as in proof of Theorem 2 in Wang and Leblanc (2008) and Theorem 1 above, we can show \( \left\| {{W_i}} \right\| \mathop {\sup }\nolimits _\varGamma {\left\| {\frac{{\partial \rho _i^{'}(\psi )}}{{\partial \psi }}} \right\| ^2} \leqslant \left\| {{W_i}} \right\| ^{1+\delta } \mathop {\sup }\nolimits _\varGamma {\left\| {\frac{{\partial \rho _i^{'}(\psi )}}{{\partial \psi }}} \right\| ^{2+\delta } } \) where \(E \left\| {{W_i}} \right\| ^{1+\delta } \mathop {\sup }\nolimits _\varGamma {\left\| {\frac{{\partial \rho _i^{'}(\psi )}}{{\partial \psi }}} \right\| ^{2+\delta } < \infty }\), for some \(\delta >0\). We also have
Using Assumption 5, Cauchy Schwarz inequality, Eq. (7) and similar argument as in proof of Theorem 2 in Wang and Leblanc (2008) and Theorem 1 above, we can show for some \(\delta >0\)
where
Define
It follows from the uniform strong law of large numbers of White and Domowitz (1984) that \(\frac{1}{N}\frac{{{\partial ^2}{Q_N}(\psi )}}{{\partial \psi \partial \psi '}} \) and \(\frac{{{\partial ^2}Q(\psi )}}{{\partial \psi \partial \psi {'}}} \) converge almost surely and uniformly for all \(\psi \) in \(\varGamma \) to the same limit. From Lemma 4 of Amemiya (1973), we conclude
which is due to the fact that
Since \({\frac{{\partial \rho _i^{'}(\psi _0 )}}{{\partial \psi _0 }}} {W_i}{\rho _i}(\psi _0 )\) are independent with zero mean, by applying Assumption 2 and 5 and Theorem 3.1. in White (1980), it follows as \(N \rightarrow \infty \),
where \(C\) is given in Eq. (3). Since \(B\) is non singular, for sufficiently large \(N\), we have
Therefore by Assumption 6 and Slutsky’s theorem, we have \(\sqrt{N} \left( {{{\hat{\psi }}_{SLSE}} - {\psi _0}} \right) \xrightarrow {L}N(\mathbf{{0}},{B^{ - 1}}C{B^{ - 1}})\).
Rights and permissions
About this article
Cite this article
Rosadi, D., Peiris, S. Second-order least-squares estimation for regression models with autocorrelated errors. Comput Stat 29, 931–943 (2014). https://doi.org/10.1007/s00180-013-0470-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-013-0470-1