Seemingly unrelated regression model with unequal size observations: computational aspects

doi:10.1016/S0167-9473(02)00146-9

Computational Statistics & Data Analysis

Volume 41, Issue 1, 28 November 2002, Pages 211-229

https://doi.org/10.1016/S0167-9473(02)00146-9 Get rights and content

Abstract

The computational solution of the seemingly unrelated regression model with unequal size observations is considered. Two algorithms to solve the model when treated as a generalized linear least-squares problem are proposed. The algorithms have as a basic tool the generalized QR decomposition (GQRD) and efficiently exploit the block-sparse structure of the matrices. One of the algorithms reduces the computational burden of the estimation procedure by not computing explicitly the RQ factorization of the GQRD. The maximum likelihood estimation of the model when the covariance matrix is unknown is also considered.

Section snippets

Seemingly unrelated regression with unequal size observations

The seemingly unrelated regression (SUR) model is defined by the set of regressions $y_{i} =X_{i} β_{i} +u_{i}, i=1,…,G,$ where $X_{i} ∈ R^{t×k_{i}}$ , $y_{i} ∈ R^{t}$ and the disturbance vector $u_{i} ∈ R^{t}$ has zero mean and variance–covariance matrix σ_i,iI_t. Furthermore, the disturbances are contemporaneously correlated across the equations, i.e. E(u_iu_j^T)=σ_i,jI_t. In the compact form the SUR model can be written as $y_{1} ⋮ y_{G} = X_{1} ⋱ X_{G} β_{1} ⋮ β_{G} + u_{1} ⋮ u_{G}$ or $vec (Y)= ⊕ i=1 G X_{i} vec ({β_{i}}_{G})+ vec (U),$ where $Y=(y_{1} ⋯ y_{G}), U=(u_{1} ⋯ u_{G})$ , the direct sum of matrices ⊕_i=1^GX_i≡⊕_iX_i≡diag(X₁

Numerical solution of the SUR-USO model

In the SUR-USO model each regression has different number of observations. That is, $y_{i},u_{i} ∈ R^{t_{i}}$ , $X_{i} ∈ R^{t_{i}×k_{i}}$ and the covariance matrices, for i<j, are given by $E(u_{i} u_{j}^{T})=σ_{ij} (I_{t_{i}} 0_{t_{i}×(t_{j}−t_{i})}),$ where it has been assumed that t_i⩽t_i+1. The compact form of the SUR-USO model is given by $vec ({y_{i}})= ⊕_{i} X_{i} vec ({β_{i}})+ vec ({u_{i}}).$ The dispersion of vec({u_i}) has a block matrix structure, where the (i,j)th block is given by (2).

Consider partitioning and reordering the observations of each regression by $y_{i} = y_{1,i} y_{2,i} ⋮ y_{i,i} h_{1} h_{2}$

Efficient solution of the GLLSP

For the efficient solution of the GLLSP (8) using the GQRD (9) the block-sparse structure of the matrices needs to be exploited. Consider first the GQRD $R ̄^{(0)} 0 =Q_{0}^{T} X ̄_{1}$ and $Q_{0}^{T} C ̄_{1,1} P_{0} = K μ_{1} −K C ̄_{1,1}^{(0)} W_{1,1} 0 W ̃_{1,1} K μ_{1} −K,$ where $C ̄_{1,1}^{(0)}$ and $W ̃_{1,1}$ are upper triangular and P₀ is orthogonal. Furthermore, $R ̄^{(0)} =⊕_{i} R_{i}^{(0)}$ and $Q_{0} =(⊕_{i} Q ̂_{0,i} ⊕_{i} Q ̃_{0,i})≡ Q ̂_{0,1} Q ̃_{0,1} ⋱ ⋱ Q ̂_{0,G} Q ̃_{0,G},$ where $X_{1,i} =(Q ̂_{0,i} Q ̃_{0,i}) R_{i}^{(0)} 0 = Q ̂_{0,i} R_{i}^{(0)}$ is the QRD of X_1,i for i=1,2,…,G. Using (11), the GLLSP (8) can be equivalently written as $arg min β, v ̂_{1}, v ̃_{1}, v ̄_{2},…,$

A recursive strategy for solving the SUR-USO model

In the GQRD (9) the computations of the QRD (9a) and RQD (9b) can be interleaved. The orthogonal matrix Q_i^T in (14a) when applied from the left of $(X ̄ C ̄)$ to annihilate $X ̄_{i}$ will fill-in a block in the lower part of C̄. This fill-in is eliminated by the application of an orthogonal transformation from the right of the modified C̄. That is, following , , $Q_{i}^{T} K μ_{i} μ_{i+1} ⋯ μ_{G} C ̄_{1,1}^{(G−i)} 0 C ̄_{1,i+1}^{(G−i)} ⋯ C ̄_{1,G}^{(G−i)} 0 C ̄_{i,i}^{(0)} 0 ⋯ 0 = K μ_{i} μ_{i+1} ⋯ μ_{G} C ̂_{1,1}^{(G−i)} C ̂_{1,i}^{(G−i)} C ̄_{1,i+1}^{(G−i+1)} ⋯ C ̄_{1,G}^{(G−i+1)} C ̂_{i,1}^{(G−i)} C ̂_{i,i}^{(G−i)} C ̄_{i,i+1}^{}$

Maximum likelihood estimation

Under normality assumptions, the maximum likelihood (ML) estimators for β_i and Σ derive from the solution of the non-linear equations $∂ L ∂β =0$ and $∂ L ∂Σ =0,$ where $L$ is the log-likelihood function for the SUR-USO model (3). The non-linear equations (29) are solved by using the EM algorithm. An initial estimator for Σ is choosen in order to obtain an estimator for β_i from (29a), which in term is used to provide a new estimator for Σ. This process is repeated until convergence (Dempster et al., 1977).

The

Conclusions

Computationally efficient methods to solve the SUR model with unequal size of observations (SUR-USO) which is treated as a GLLSP have been proposed. The algorithms use the GQRD to solve the GLLSP by exploiting the block-sparse structure of the matrices. The first algorithm initially computes the QRD of the exogenous matrix by annihilating from bottom to the top blocks of observations which consist of a non-zero block-superdiagonal. The annihilation of the blocks is obtained by orthogonal

Acknowledgements

The authors are grateful to Jesse Barlow, Jesse Barlow and the anonymous referees for their constructive comments and suggestions.

References (15)

P. Schmidt
Estimation of seemingly unrelated regressions with unequal numbers of observations
J. Econometrics
(1977)
A.P. Dempster et al.
Maximum likelihood from incomplete data via the EM algorithm. Discussion
J. Roy. Statist. Soc. Ser. B
(1977)
Foschi, P., Kontoghiorghes, E.J., 2002. Estimation of VAR(p) models: computational aspects. Computat. Econom., in...
Foschi, P., Belsley, D.A., Kontoghiorghes, E.J. A comparative study of algorithms for solving seemingly unrelated...
G.H. Golub et al.
Matrix Computations
(1996)
R.R. Hocking et al.
Estimation of parameters in the multivariate normal distribution with missing observations
J. Amer. Statist. Assoc.
(1968)
E.J. Kontoghiorghes
Parallel strategies for computing the orthogonal factorizations used in the estimation of econometric models
Algorithmica
(1999)

There are more references available in the full text version of this article.

Cited by (0)

^☆: This work is in part supported by the Swiss National Foundation Grants 1214-056900.99/1 and 2000-061875.00/1. Part of the work of the second author was done while he was visiting INRIA-IRISA, Rennes, France under the support of the host institution and the Swiss National Foundation Grant 83R-065887.

View full text

Seemingly unrelated regression model with unequal size observations: computational aspects☆

Abstract

Section snippets

Seemingly unrelated regression with unequal size observations

Numerical solution of the SUR-USO model

Efficient solution of the GLLSP

A recursive strategy for solving the SUR-USO model

Maximum likelihood estimation

Conclusions

Acknowledgements

J. Econometrics

Maximum likelihood from incomplete data via the EM algorithm. Discussion

J. Roy. Statist. Soc. Ser. B

Matrix Computations

Estimation of parameters in the multivariate normal distribution with missing observations

J. Amer. Statist. Assoc.

Parallel strategies for computing the orthogonal factorizations used in the estimation of econometric models

Algorithmica