Skip to main content
SpringerLink
Log in

Multiple Change Points Detection in High-Dimensional Multivariate Regression

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

This paper considers the problem of detecting structural changes in a high-dimensional regression setting. The structural parameters are subject to abrupt changes of unknown magnitudes at unknown locations. The authors propose a new procedure that minimizes a penalized least-squares loss function via a dynamic programming algorithm for estimating the locations of change points. To alleviate the computational burden, the authors adopt a prescreening procedure by eliminating a large number of irrelevant points before implementing estimation procedure. The number of change points is determined via Schwarz’s information criterion. Under mild assumptions, the authors establish the consistency of the proposed estimators, and further provide error bounds for estimated parameters which achieve almost-optimal rate. Simulation studies show that the proposed method performs reasonably well in terms of estimation accuracy, and a real data example is used for illustration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bai J, Common breaks in beans and variances for panel data, Journal of Econometrics, 2010, 157: 78–92.

    Article  MathSciNet  MATH  Google Scholar 

  2. Lee S, Seo M H, and Shin Y, The lasso for high dimensional regression with a possible change point, Journal of the Royal Statistical Society, Series B, 2016, 78: 193–210.

    Article  MathSciNet  MATH  Google Scholar 

  3. Kaul A, Jandhyala V, and Fotopoulos S, An efficient two step algorithm for high dimensional change point regression models without grid search, Journal of Machine Research, 2019, 20: 1–40.

    MathSciNet  MATH  Google Scholar 

  4. Yuan M, Ekici A, Lu Z, et al., Dimension reduction and coefficient estimation in multivariate linear regression, Journal of the Royal Statistical Society, Series B, 2007, 69: 329–346.

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen K, Dong H, and Chan K S, Reduced rank regression via adaptive nuclear norm penalization, Biometrika, 2013, 100(4): 901–920.

    Article  MathSciNet  MATH  Google Scholar 

  6. Bing X and Wegkamp M H, Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models, The Annals of Statistics, 2019, 47: 3157–3184.

    Article  MathSciNet  MATH  Google Scholar 

  7. Raskutti G, Yuan M, and Chen H, Convex regularization for high-dimensional multiresponse tensor regression, The Annals of Statistics, 2019, 47: 1554–1584.

    Article  MathSciNet  MATH  Google Scholar 

  8. Zou C, Ke Y, and Zhang W, Estimation of low rank high dimensional multivariate linear models for multi-response data, Journal of the American Statistical Association, 2022, 117: 693–703.

    Article  MathSciNet  Google Scholar 

  9. Chen K, Chan K S, and Stenseth N C, Reduced rank stochastic regression with a sparse singular value decomposition, Journal of the Royal Statistical Society, Series B, 2012, 74: 203–221.

    Article  MathSciNet  MATH  Google Scholar 

  10. Leonardi F and Bühlmann P, Computationally efficient change point detection for high-dimensional regression, arXiv: 1601.03704, 2016.

  11. Zhang B, Geng J, and Lai L, Multiple change-points estimation in linear regression models via sparse group lasso, IEEE Trans. Signal Processing, 2015, 63(9): 2209–2224.

    Article  MathSciNet  MATH  Google Scholar 

  12. Wang D, Lin K, and Willett R, Statistically and computationally efficient change point localization in regression settings, arXiv: 1906.11364v1, 2019.

  13. Candès E J and Recht B, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 2009, 9: 717–772.

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhou H and Li L, Regularized matrix regression, Journal of the Royal Statistical Society, Series B, 2014, 76: 463–483.

    Article  MathSciNet  MATH  Google Scholar 

  15. Nesterov Y, A method of solving a convex programming problem with convergence rate O(1/k2), Soviet Mathematics Doklady, 1983, 27: 372–376.

    MATH  Google Scholar 

  16. Beck A and Teboulle M, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, 2009, 2: 183–202.

    Article  MathSciNet  MATH  Google Scholar 

  17. Bai J and Perron P, Computation and analysis of multiple structural changes models, Journal of Applied Econometrics, 2003, 18: 1–22.

    Article  Google Scholar 

  18. Killick R, Fearnhead P, and Eckley I A, Optimal detection of changepoints with a linear computational cost, Journal of the American Statistical Association, 2012, 107: 1590–1598.

    Article  MathSciNet  MATH  Google Scholar 

  19. Scott A G and Knott M, A cluster analysis method for grouping means in the analysis of variance, Biometrics, 1974, 30: 507–512.

    Article  MATH  Google Scholar 

  20. Fryzlewicz P, Wild binary segmentation for multiple change-point detection, The Annals of Statistics, 2014, 42: 2243–2281.

    Article  MathSciNet  MATH  Google Scholar 

  21. Negahban S and Wainwright M J, Estimation of (near) low-rank matrices with noise and high-dimensional scaling, The Annals of Statistics, 2011, 39: 1069–1097.

    Article  MathSciNet  MATH  Google Scholar 

  22. Bickel P, Ritov Y, and Tsybakov A, Simultaneous analysis of lasso and Dantzig selector, The Annals of Statistics, 2009, 37: 1705–1732.

    Article  MathSciNet  MATH  Google Scholar 

  23. Negahban S N, Ravikumar P, Wainwright M J, et al., A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers, Statistical Science, 2012, 27: 538–557.

    Article  MathSciNet  MATH  Google Scholar 

  24. Wainwright M J, High-Dimensional Statistics: A Non-Asymptotic Viewpoint, Cambridge University Press, Cambridge, 2019.

    Book  MATH  Google Scholar 

  25. Reinsel G C and Velu R P, Multivariate Reduced Rank Regression Theory and Applications, Springer, New York, 1998.

    Book  MATH  Google Scholar 

  26. Chen K, Dong H, and Chan K, Reduced rank regression via adaptive nuclear norm penalization, Biometrika, 2013, 100(4): 901–920.

    Article  MathSciNet  MATH  Google Scholar 

  27. Boysen L, Kempe A, Liebscher V, et al., Consistencies and rates of convergence of jump-penalized least squares estimators, The Annals of Statistics, 2009, 37: 157–183.

    Article  MathSciNet  MATH  Google Scholar 

  28. Zou C, Yin G, Feng L, et al., Nonparametric maximum likelihood approach to multiple change-point problems, The Annals of Statistics, 2014, 42: 970–1002.

    Article  MathSciNet  MATH  Google Scholar 

  29. Recht B, Fazel M, and Parrilo P A, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Review, 2010, 52: 471–501.

    Article  MathSciNet  MATH  Google Scholar 

  30. Vershynin R, Introduction to the non-asymptotic analysis of random matrices, arXiv: 1011.3027, 2010.

Download references

Acknowledgements

The authors have contributed equally to this work and are listed in alphabetical order.

Author information

Authors and Affiliations

  1. School of Statistics and Data Science, LPMC and KLMDASR, Nankai University, Tianjin, 300071, China

    Xiaoyan Ma

  2. School of Mathematics and Statistics, Jiangsu Normal University, Xuzhou, 221116, China

    Qin Zhou

  3. School of Science, Tianjin University of Technology and Education, Tianjin, 300222, China

    Xuemin Zi

Authors
  1. Xiaoyan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuemin Zi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuemin Zi.

Additional information

This research was supported by the National Nature Science Foundation of China under Grant Nos. 11771332, 11771220, 11671178, 11925106, 11971247, and the Nature Science Foundation of Tianjin under Grant No. 18JCJQJC46000. Ma was also supported by the Fundamental Research Funds for the Central Universities.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, X., Zhou, Q. & Zi, X. Multiple Change Points Detection in High-Dimensional Multivariate Regression. J Syst Sci Complex 35, 2278–2301 (2022). https://doi.org/10.1007/s11424-022-1205-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-022-1205-6

Keywords

Search

Navigation