Abstract
A Multivariate Regression Model Based on the Optimal Partition of Predictors (MRBOP) useful in applications in the presence of strongly correlated predictors is presented. Such classes of predictors are synthesized by latent factors, which are obtained through an appropriate linear combination of the original variables and are forced to be weakly correlated. Specifically, the proposed model assumes that the latent factors are determined by subsets of predictors characterizing only one latent factor. MRBOP is formalized in a least squares framework optimizing a penalized quadratic objective function through an alternating least-squares (ALS) algorithm. The performance of the methodology is evaluated on simulated and real data sets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi, H.: Partial least squares regression and projection on latent structure regression (PLS-regression). Wiley Interdiscip. Rev.: Comput. Stat. 2, 97–106 (2010)
Abraham, B., Merola, G.: Dimensionality reduction approach to multivariate prediction. Comput. Stat. Data Anal. 48(1), 5–16 (2005)
Anderson, T.W.: Estimating linear restrictions on regression coefficients for multivariate distributions. Ann. Math. Stat. 22, 327–351 (1951)
Bougeard, S., Hanafi, M., Qannari, E.M.: Multiblock latent root regression: application to epidemiological data. Comput. Stat. 22(2), 209–222 (2007)
Bougeard, S., Hanafi, M., Qannari, E.M.: Continuum redundancy-PLS regression: a simple continuum approach. Comput. Stat. Data Anal. 52(7), 3686–3696 (2008)
Chauvin, C., Buuvrel, I., Belceil, P.A., Orand, J.P., Guillemot, D., Sanders, P.: A pharmaco-epidemiological analysis of factors associated with antimicrobial consumption level in turkey broiler flocks. Comput. Stat. Data Anal. 36, 199–211 (2005)
De Jong, S.: SIMPLS: an alternative approach to partial least squares regression. Chemom. Intell. Lab. Syst. 18, 251–263 (1993)
Escoufier, Y.: Le traitement des variables vectorielles. Biometrics 29, 751–760 (1973)
Frank, I.E., Friedman, J.: A statistical view of some chemometrics regression tools. Technometrics 35, 109–148 (1993)
Hocking, R.R.: The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12, 55–67 (1970)
Hotelling, H.: The most predictable criterion. J. Educ. Psychol. 25, 139–142 (1935)
Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 198, 193–218 (1985)
Izenman, A.J.: Reduced-rank regression for the multivariate linear model. J. Multivar. Anal. 5, 248–262 (1975)
Jolliffe, I.T.: A note on the use of principal components in regression. J. R. Stat. Soc., Ser. C, Appl. Stat. 31(3), 300–303 (1982)
Krzanowski, W.J.: Principles of Multivariate Analysis: A User’s Perspective. Oxford University Press, London (2000)
Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Subspace, Latent Structure and Feature Selection, vol. 3940, pp. 34–51 (2006)
Stone, M.: Cross-validation choice and assessment of statistical predictions. J. R. Stat. Soc. B 36, 111–147 (1974)
Stone, M., Brooks, R.J.: Continuum regression: cross-validated sequentially constructed prediction embracing ordinary least squares, partial least squares and principal components regression. J. R. Stat. Soc. B 52(2), 237–269 (1990)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58(1), 267–288 (1996)
Tutz, G., Ulbricht, J.: Penalized regression with correlation-based penalty. Stat. Comput. 19(1), 239–253 (2009)
Van Den Wollenberg, A.L.: Redundancy analysis an alternative for canonical correlation analysis. Psychometrika 42(2), 207–219 (1977)
Waldro, L., Pintilie, M., Tsao, M.S., Shepherd, F.A., Huttenhower, C., Jurisica, I.: Optimized application of penalized regression methods to diverse genomic data. Bioinformatics 27(24), 3399–3406 (2011)
Witten, D.M., Tibshirani, R.: Covariance-regularized regression and classification for high dimensional problems. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71, 615–636 (2009)
Wold, H.: Estimation of principal components and related models by iterative least squares. In: Krishnaiaah, P.R. (ed.) Multivariate Analysis, pp. 391–420. Academic Press, New York (1966)
Yuan, M., Ekici, A., Lu, Z., Monteiro, R.: Dimension reduction and coefficient estimation in multivariate linear regression. J. R. Stat. Soc., Ser. B, Stat. Methodol. 69, 329–346 (2007)
Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. B 68, 49–67 (2006)
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301–320 (2005)
Acknowledgements
The authors are grateful to the editor and anonymous referees of Statistics and Computing for their valuable comments and suggestions which improved the clarity and the relevance of the first version.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Martella, F., Vicari, D. & Vichi, M. Partitioning predictors in multivariate regression models. Stat Comput 25, 261–272 (2015). https://doi.org/10.1007/s11222-013-9430-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-013-9430-4