Skip to main content
SpringerLink
Log in

Change Point Detection in Periodic Panel Data Using a Mixture-Model-Based Approach

  • Conference paper
  • First Online:
Trends and Applications in Knowledge Discovery and Data Mining (PAKDD 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11607))

Included in the following conference series:

  • 809 Accesses

Abstract

This paper describes a novel method for common change detection in panel data emanating from smart electricity and water networks. The proposed method relies on a representation of the data by classes whose probabilities of occurrence evolve over time. This dynamics is assumed to be piecewise periodic due to the cyclic nature of the studied data, which allows the detection of change points. Our strategy is based on a hierarchical mixture of t-distributions which entails some robustness properties. The parameter estimation is performed using an incremental strategy, which has the advantage to allow the processing of large datasets. The experiments carried out on realistic data showed the full relevance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M.L., et al.: Predictive classification of water consumption time series using non-homogeneous Markov models. In: IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 323–331 (2017)

    Google Scholar 

  2. Bai, J.: Common breaks in means and variances for panel data. J. Econ. 157(1), 78–92 (2010)

    Article  MathSciNet  Google Scholar 

  3. Bai, J., Carrion-I-Silvestre, J.L.: Structural changes, common stochastic trends, and unit roots in panel data. Rev. Econ. Stud. 76(2), 471–501 (2009)

    Article  MathSciNet  Google Scholar 

  4. Chan, J., Horváth, L., Hušková, M.: Darling-Erdős limit results for change-point detection in panel data. J. Stat. Plann. Infer. 143(5), 955–970 (2013)

    Article  Google Scholar 

  5. Cho, H., et al.: Change-point detection in panel data via double CUSUM statistic. Electron. J. Stat. 10(2), 2000–2038 (2016)

    Article  MathSciNet  Google Scholar 

  6. Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  7. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. B 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  8. Green, P.: Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. J. Roy. Stat. Soc. B 46(2), 149–192 (1984)

    MathSciNet  MATH  Google Scholar 

  9. Horváth, L., Hušková, M.: Change-point detection in panel data. J. Time Ser. Anal. 33(4), 631–648 (2012)

    Article  MathSciNet  Google Scholar 

  10. Im, K.S., Lee, J., Tieslau, M.: Panel LM unit-root tests with level shifts. Oxf. Bull. Econ. Stat. 67(3), 393–419 (2005)

    Article  Google Scholar 

  11. Joseph, L., Wolfson, D.B.: Estimation in multi-path change-point problems. Commun. Stat. Theory Methods 21(4), 897–913 (1992)

    Article  Google Scholar 

  12. Joseph, L., Wolfson, D.B.: Maximum likelihood estimation in the multi-path change-point problem. Ann. Inst. Stat. Math. 45(3), 511–530 (1993)

    Article  MathSciNet  Google Scholar 

  13. Krishnapuram, B., Carin, L., Figueiredo, M.A.T., Hartemink, A.J.: Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 957–968 (2005)

    Article  Google Scholar 

  14. Li, F., Tian, Z., Xiao, Y., Chen, Z.: Variance change-point detection in panel data models. Econ. Lett. 126, 140–143 (2015)

    Article  MathSciNet  Google Scholar 

  15. McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (2008)

    Book  Google Scholar 

  16. McLachlan, G.J., Ng, S.K., Bean, R.: Robust cluster analysis via mixture models. Austrian J. Stat. 35(2&3), 157–174 (2016)

    Google Scholar 

  17. Neal, R., Hinton, G.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models, vol. 89. Kluwer, Dordrecht (1998)

    Google Scholar 

  18. Samé, A., Chamroukhi, F., Govaert, G., Aknin, P.: Model-based clustering and segmentation of time series with changes in regime. Adv. Data Anal. Classif. 5(4), 301–321 (2011)

    Article  MathSciNet  Google Scholar 

  19. Wolfe, J., Haghighi, A., Klein, D.: Fully distributed EM for very large datasets. In: Proceedings of the 25th International Conference on Machine Learning, ICML 2008, pp. 1184–1191. ACM (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IFSTTAR, COSYS, GRETTIA, Université Paris-Est, 77447, Marne la Vallée, France

    Allou Samé & Milad Leyli-Abadi

Authors
  1. Allou Samé
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Milad Leyli-Abadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Allou Samé .

Editor information

Editors and Affiliations

  1. University of Macau, Macao, China

    Leong Hou U.

  2. Singapore Management University, Singapore, Singapore

    Hady W. Lauw

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Samé, A., Leyli-Abadi, M. (2019). Change Point Detection in Periodic Panel Data Using a Mixture-Model-Based Approach. In: U., L., Lauw, H. (eds) Trends and Applications in Knowledge Discovery and Data Mining. PAKDD 2019. Lecture Notes in Computer Science(), vol 11607. Springer, Cham. https://doi.org/10.1007/978-3-030-26142-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-26142-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-26141-2

  • Online ISBN: 978-3-030-26142-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation