Abstract
This paper considers the problem of detecting structural changes in a high-dimensional regression setting. The structural parameters are subject to abrupt changes of unknown magnitudes at unknown locations. The authors propose a new procedure that minimizes a penalized least-squares loss function via a dynamic programming algorithm for estimating the locations of change points. To alleviate the computational burden, the authors adopt a prescreening procedure by eliminating a large number of irrelevant points before implementing estimation procedure. The number of change points is determined via Schwarz’s information criterion. Under mild assumptions, the authors establish the consistency of the proposed estimators, and further provide error bounds for estimated parameters which achieve almost-optimal rate. Simulation studies show that the proposed method performs reasonably well in terms of estimation accuracy, and a real data example is used for illustration.
Similar content being viewed by others
References
Bai J, Common breaks in beans and variances for panel data, Journal of Econometrics, 2010, 157: 78–92.
Lee S, Seo M H, and Shin Y, The lasso for high dimensional regression with a possible change point, Journal of the Royal Statistical Society, Series B, 2016, 78: 193–210.
Kaul A, Jandhyala V, and Fotopoulos S, An efficient two step algorithm for high dimensional change point regression models without grid search, Journal of Machine Research, 2019, 20: 1–40.
Yuan M, Ekici A, Lu Z, et al., Dimension reduction and coefficient estimation in multivariate linear regression, Journal of the Royal Statistical Society, Series B, 2007, 69: 329–346.
Chen K, Dong H, and Chan K S, Reduced rank regression via adaptive nuclear norm penalization, Biometrika, 2013, 100(4): 901–920.
Bing X and Wegkamp M H, Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models, The Annals of Statistics, 2019, 47: 3157–3184.
Raskutti G, Yuan M, and Chen H, Convex regularization for high-dimensional multiresponse tensor regression, The Annals of Statistics, 2019, 47: 1554–1584.
Zou C, Ke Y, and Zhang W, Estimation of low rank high dimensional multivariate linear models for multi-response data, Journal of the American Statistical Association, 2022, 117: 693–703.
Chen K, Chan K S, and Stenseth N C, Reduced rank stochastic regression with a sparse singular value decomposition, Journal of the Royal Statistical Society, Series B, 2012, 74: 203–221.
Leonardi F and Bühlmann P, Computationally efficient change point detection for high-dimensional regression, arXiv: 1601.03704, 2016.
Zhang B, Geng J, and Lai L, Multiple change-points estimation in linear regression models via sparse group lasso, IEEE Trans. Signal Processing, 2015, 63(9): 2209–2224.
Wang D, Lin K, and Willett R, Statistically and computationally efficient change point localization in regression settings, arXiv: 1906.11364v1, 2019.
Candès E J and Recht B, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 2009, 9: 717–772.
Zhou H and Li L, Regularized matrix regression, Journal of the Royal Statistical Society, Series B, 2014, 76: 463–483.
Nesterov Y, A method of solving a convex programming problem with convergence rate O(1/k2), Soviet Mathematics Doklady, 1983, 27: 372–376.
Beck A and Teboulle M, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, 2009, 2: 183–202.
Bai J and Perron P, Computation and analysis of multiple structural changes models, Journal of Applied Econometrics, 2003, 18: 1–22.
Killick R, Fearnhead P, and Eckley I A, Optimal detection of changepoints with a linear computational cost, Journal of the American Statistical Association, 2012, 107: 1590–1598.
Scott A G and Knott M, A cluster analysis method for grouping means in the analysis of variance, Biometrics, 1974, 30: 507–512.
Fryzlewicz P, Wild binary segmentation for multiple change-point detection, The Annals of Statistics, 2014, 42: 2243–2281.
Negahban S and Wainwright M J, Estimation of (near) low-rank matrices with noise and high-dimensional scaling, The Annals of Statistics, 2011, 39: 1069–1097.
Bickel P, Ritov Y, and Tsybakov A, Simultaneous analysis of lasso and Dantzig selector, The Annals of Statistics, 2009, 37: 1705–1732.
Negahban S N, Ravikumar P, Wainwright M J, et al., A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers, Statistical Science, 2012, 27: 538–557.
Wainwright M J, High-Dimensional Statistics: A Non-Asymptotic Viewpoint, Cambridge University Press, Cambridge, 2019.
Reinsel G C and Velu R P, Multivariate Reduced Rank Regression Theory and Applications, Springer, New York, 1998.
Chen K, Dong H, and Chan K, Reduced rank regression via adaptive nuclear norm penalization, Biometrika, 2013, 100(4): 901–920.
Boysen L, Kempe A, Liebscher V, et al., Consistencies and rates of convergence of jump-penalized least squares estimators, The Annals of Statistics, 2009, 37: 157–183.
Zou C, Yin G, Feng L, et al., Nonparametric maximum likelihood approach to multiple change-point problems, The Annals of Statistics, 2014, 42: 970–1002.
Recht B, Fazel M, and Parrilo P A, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 2010, 52: 471–501.
Vershynin R, Introduction to the non-asymptotic analysis of random matrices, arXiv: 1011.3027, 2010.
Acknowledgements
The authors have contributed equally to this work and are listed in alphabetical order.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by the National Nature Science Foundation of China under Grant Nos. 11771332, 11771220, 11671178, 11925106, 11971247, and the Nature Science Foundation of Tianjin under Grant No. 18JCJQJC46000. Ma was also supported by the Fundamental Research Funds for the Central Universities.
Rights and permissions
About this article
Cite this article
Ma, X., Zhou, Q. & Zi, X. Multiple Change Points Detection in High-Dimensional Multivariate Regression. J Syst Sci Complex 35, 2278–2301 (2022). https://doi.org/10.1007/s11424-022-1205-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-022-1205-6