Skip to main content
SpringerLink
Log in

Finder: A novel approach of change point detection for multivariate time series

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The multivariate time series often contain complex mixed inputs, with complex correlations between them. Detecting change points in multivariate time series is of great importance, which can find anomalies early and reduce losses, yet very challenging as it is affected by many complex factors, i.e., dynamic correlations and external factors. The performance of traditional methods typically scales poorly. In this paper, we propose Finder, a novel approach of change point detection via multivariate fusion attention networks. Our model consists of two key modules. First, in the time series prediction module, we employ multi-level attention networks based on the Transformer and integrate the external factor fusion component, achieving feature extraction and fusion of multivariate data. Secondly, in the change point detection module, a deep learning classifier is used to detect change points, improving efficiency and accuracy. Extensive experiments prove the superiority and effectiveness of Finder on two real-world datasets. Our approach outperforms the state-of-the-art methods by up to 10.50% on the F1 score.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://www.kaggle.com/c/global-energy-forecasting-competition-2012-load-forecasting/data

  2. http://archive.ics.uci.edu/ml/datasets/Beijing+Multi-Site+Air-Quality+Data

References

  1. Adams RP, MacKay DJ (2007) Bayesian online changepoint detection. Stat 1050:19

    Google Scholar 

  2. Alaa AM, van der Schaar M (2019) Attentive state-space modeling of disease progression. In: Advances in neural information processing systems, pp 11,338–11,348

  3. Aminikhanghahi S, Cook DJ (2017) A survey of methods for time series change point detection. Knowl Inf Syst 51(2):339– 367

    Article  Google Scholar 

  4. Asteriou D, Hall SG (2011) Arima models and the box–jenkins methodology. Appl Econom 2(2):265–286

    Google Scholar 

  5. Ba J, Kiros JR, Hinton GE (2016) Layer normalization. Stat 1050:21

    Google Scholar 

  6. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd International conference on learning representations, ICLR 2015

  7. Bouktif S, Fiaz A, Ouni A, Serhani MA (2020) Multi-sequence lstm-rnn deep learning and metaheuristics for electric load forecasting. Energies 13(2):391

    Article  Google Scholar 

  8. Canizo M, Triguero I, Conde A, Onieva E (2019) Multi-head cnn–rnn for multi-time series anomaly detection: An industrial case study. Neurocomputing 363:246–260

    Article  Google Scholar 

  9. Chalapathy R, Chawla S (2019) Deep learning for anomaly detection: Asurvey

  10. Chandola V, Vatsavai RR (2011) A gaussian process based online change detection algorithm for monitoring periodic time series. In: Proceedings of the 2011 SIAM international conference on data mining, pp. 95–106. SIAM

  11. Clevert DA, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). Computer ence

  12. Dadashova B, Li X, Turner S, Koeneman P (2021) Multivariate time series analysis of traffic congestion measures in urban areas as they relate to socioeconomic indicators. Socio-Economic Planning Sciences 75:100877

    Article  Google Scholar 

  13. Dauphin YN, Fan A, Auli M, Grangier D (2017) Language modeling with gated convolutional networks. In: International conference on machine learning, pp 933–941

  14. Fan C, Zhang Y, Pan Y, Li X, Zhang C, Yuan R, Wu D, Wang W, Pei J, Huang H (2019) Multi-horizon time series forecasting with temporal attention learning. In: Proceedings of the 25th ACM SIGKDD International conference on knowledge discovery & data mining, pp 2527–2535

  15. Gal Y, Ghahramani Z (2016) A theoretically grounded application of dropout in recurrent neural networks. In: Advances in neural information processing systems, pp 1019–1027

  16. Guo T, Lin T, Antulov-Fantulin N (2019) Exploring interpretable lstm neural networks over multi-variable data. In: International conference on machine learning, pp 2494–2504

  17. Hayashi T, Fujita H (2021) Cluster-based zero-shot learning for multivariate data. J Ambient Intell Human Comput 12(2):1897–1911

    Article  Google Scholar 

  18. Hernandez-Matamoros A, Fujita H, Hayashi T, Perez-Meana H (2020) Forecasting of covid19 per regions using arima models and polynomial functions. Appl Soft Comput 96(106):610

    Google Scholar 

  19. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  20. Hong T, Pinson P, Fan S, Zareipour H, Troccoli A, Hyndman RJ (2016) Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond

  21. Huang S, Wang D, Wu X, Tang A (2019) Dsanet: Dual self-attention network for multivariate time series forecasting. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2129–2132

  22. Hundman K, Constantinou V, Laporte C, Colwell I, Soderstrom T (2018) Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 387–395

  23. Kawahara Y, Sugiyama M (2012) Sequential change-point detection based on direct density-ratio estimation. Stat Anal Data Min 5(2):114–127

    Article  MathSciNet  Google Scholar 

  24. Kim TY, Cho SB (2018) Web traffic anomaly detection using c-lstm neural networks. Expert Syst Appl 106:66–76

    Article  Google Scholar 

  25. Kingma D, Ba J (2014) Adam: A method for stochastic optimization. Computer Science

  26. LeCun Y, Bengio Y, et al. (1995) Convolutional networks for images, speech, and time series. Handbook Brain Theory Neural Netw 3361(10):1995

    Google Scholar 

  27. Li S, Jin X, Xuan Y, Zhou X, Chen W, Wang Y, Yan X (2019) Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In: Advances in Neural information processing systems, pp 5243–5253

  28. Liang Y, Ke S, Zhang J, Yi X, Zheng Y (2018) Geoman: Multi-level attention networks for geo-sensory time series prediction. In: IJCAI, pp 3428–3434

  29. Lv Z, Xu J, Zheng K, Yin H, Zhao P, Zhou X (2018) Lc-rnn: a deep learning model for traffic speed prediction. In: Proceedings of the 27th international joint conference on artificial intelligence, pp 3470–3476

  30. Makridakis S, Spiliotis E, Assimakopoulos V (2020) The m4 competition: 100,000 time series and 61 forecasting methods. Int J Forecast 36(1):54–74

    Article  Google Scholar 

  31. Munir M, Siddiqui SA, Dengel A, Ahmed S (2018) Deepant: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access 7:1991–2005

    Article  Google Scholar 

  32. Pan Z, Liang Y, Wang W, Yu Y, Zheng Y, Zhang J (2019) Urban traffic prediction from spatio-temporal data using deep meta learning. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1720–1730

  33. Qin Y, Song D, Cheng H, Cheng W, Jiang G, Cottrell GW (2017) A dual-stage attention-based recurrent neural network for time series prediction. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 2627–2633

  34. Qiu J, Jammalamadaka SR, Ning N (2020) Multivariate time series analysis from a bayesian machine learning perspective. Ann Math Artif Intell 88(10):1061–1082

    Article  MathSciNet  Google Scholar 

  35. Rangapuram SS, Seeger MW, Gasthaus J, Stella L, Wang Y, Januschowski T (2018) Deep state space models for time series forecasting. In: Advances in neural information processing systems, pp 7785–7794

  36. Ribeiro GT, Mariani VC, dos Santos Coelho L (2019) Enhanced ensemble structures using wavelet neural networks applied to short-term load forecasting. Eng Appl Artif Intel 82:272–281

    Article  Google Scholar 

  37. Sadouk L (2018) Cnn approaches for time series classification. In: Time series analysis-data, methods, and applications. IntechOpen

  38. Salinas D, Flunkert V, Gasthaus J, Januschowski T (2019) Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting

  39. Sezer OB, Gudelek MU, Ozbayoglu AM (2020) Financial time series forecasting with deep learning: A systematic literature review: 2005–2019. Appl Soft Comput 90(106):181

    Google Scholar 

  40. Silva L (2014) A feature engineering approach to wind power forecasting: Gefcom 2012. Int J Forecast 30(2):395–401

    Article  Google Scholar 

  41. Song H, Rajan D, Thiagarajan JJ, Spanias A (2018) Attend and diagnose: Clinical time series analysis using attention models. In: 32nd AAAI Conference on artificial intelligence, AAAI 2018, pp. 4091–4098. AAAI press

  42. Su Y, Zhao Y, Niu C, Liu R, Sun W, Pei D (2019) Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2828–2837

  43. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  44. Taylor SJ, Letham B (2018) Forecasting at scale. Am Stat 72(1):37–45

    Article  MathSciNet  Google Scholar 

  45. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  46. Wang D, Zhang J, Cao W, Li J, Zheng Y (2018) When will you arrive? estimating travel time based on deep neural networks. In: AAAI, vol 18, pp 1–8

  47. Wang Y, Smola A, Maddix D, Gasthaus J, Foster D, Januschowski T (2019) Deep factors for forecasting. In: International conference on machine learning, pp 6607–6617

  48. Wei WW (2018) Multivariate time series analysis and applications. John Wiley Sons, New York

    Google Scholar 

  49. Wen R, Torkkola K, Narayanaswamy BM (2017) A multi-horizon quantile recurrent forecaster. Stat 1050:29

    Google Scholar 

  50. Xu H, Chen W, Zhao N, Li Z, Bu J, Li Z, Liu Y, Zhao Y, Pei D, Feng Y et al (2018) Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In: Proceedings of the 2018 world wide web conference, pp 187–196

  51. Zhang A, Paisley J (2018) Deep bayesian nonparametric tracking. In: International conference on machine learning, pp 5833–5841

  52. Zhang J, Zheng Y, Qi D (2017) Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol 31

  53. Zheng J, Xu C, Zhang Z, Li X (2017) Electric load forecasting in smart grids using long-short-term-memory based recurrent neural network. In: 2017 51st Annual conference on information sciences and systems (CISS), pp 1–6. IEEE

  54. Zheng Y, Liu Q, Chen E, Ge Y, Zhao JL (2014) Time series classification using multi-channels deep convolutional neural networks. In: International conference on web-age information management, pp 298–310. Springer

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shanghai University of Electric Power Shanghai, Shanghai, 200090, China

    Haizhou Du & Ziyi Duan

Authors
  1. Haizhou Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziyi Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haizhou Du.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Du, H., Duan, Z. Finder: A novel approach of change point detection for multivariate time series. Appl Intell 52, 2496–2509 (2022). https://doi.org/10.1007/s10489-021-02532-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02532-x

Keywords

Search

Navigation