Skip to main content
SpringerLink
Log in

Anomaly detection via a combination model in time series data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Since the time series data have the characteristics of a large amount of data and non-stationarity, we usually cannot obtain a satisfactory result by a single-model-based method to detect anomalies in time series data. To overcome this problem, in this paper, a combination-model-based approach is proposed by combining a similarity-measurement-based method and a model-based method for anomaly detection. First, the process of data representation is performed to generate a new data form to arrive at the purpose of reducing data volume. Furthermore, due to the anomalies being generally caused by changes in amplitude and shape, we take both the original time series data and their amplitude change data into consideration of the process of data representation to capture the shape and morphological features. Then, the results of data representation are employed to establish a model for anomaly detection. Compared with the state-of-the-art methods, experimental studies on a large number of datasets show that the proposed method can significantly improve the performance of anomaly detection with higher data anomaly resolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. McAfee A, Brynjolfsson E, Davenport TH (2012) Big data: the management revolution. Harv Bus Rev 90(10):60–68

    Google Scholar 

  2. Esling P, Agon C (2012) Time-series data mining. ACM Computing Surveys (CSUR) 45(1):1–34

    Article  Google Scholar 

  3. Wilson SJ (2017) Data representation for time series data mining: time domain approaches. Wiley Interdisciplinary Reviews: Computational Statistics 9(1):e1392

    Article  MathSciNet  Google Scholar 

  4. Keogh E, Lin J, Fu A (2004) HOT SAX: finding the most unusual time series subsequence: algorithms and applications. In proceedings of the Proc. of the 5th IEEE Int. Conf. On data mining: 440-449

  5. Chandra R, Chand S (2016) Evaluation of co-evolutionary neural network architectures for time series prediction with mobile application in finance. Appl Soft Comput 49:462–473

    Article  Google Scholar 

  6. Amini MH, Kargarian A, Karabasoglu O (2016) ARIMA-based decoupled time series forecasting of electric vehicle charging demand for stochastic power system operation. Electr Power Syst Res 140:378–390

    Article  Google Scholar 

  7. Zhao J, Liu K, Wang W, Liu Y (2014) Adaptive fuzzy clustering based anomaly data detection in energy system of steel industry. Inf Sci 259(3):335–345

    Article  Google Scholar 

  8. Thirtle CG, Schimmelpfennig DE, Townsend RE (2002) Induced innovation in United States agriculture, 1880–1990: time series tests and an error correction model. Am J Agric Econ 84(3):598–614

    Article  Google Scholar 

  9. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM computing surveys (CSUR) 41(3):1–58

    Article  Google Scholar 

  10. Shi Y, Li R, Zhang Y (2015) An immunity-based time series prediction approach and its application for network security situation. Intell Serv Robot 8(1):1–22

    Article  Google Scholar 

  11. Martí L, Sanchez-Pi N (2015) Anomaly detection based on sensor data in petroleum industry applications. Sensors 15(2):2774–2797

    Article  Google Scholar 

  12. Izakian H, Pedrycz W (2013) Anomaly detection in time series data using a fuzzy c-means clustering. In proceedings of the 2013 joint IFSA world congress and NAFIPS annual meeting (IFSA/NAFIPS): 1513-1518

  13. Lin J, Khade R, Li Y (2012) Rotation-invariant similarity in time series using bag-of-patterns representation. J Intell Inf Syst 39(2):287–315

    Article  Google Scholar 

  14. Xu K, Xia M, Mu X (2018) Ensemblelens: ensemble-based visual exploration of anomaly detection algorithms with multidimensional data. IEEE Trans Vis Comput Graph 25(1):109–119

    Article  Google Scholar 

  15. Han XJ, Zhang XL, Chen YY (2014) Wind power prediction model based on the combination of gray theory and time series forecasting methods. In Proceedings of the Applied Mechanics and Materials 448:1721–1726

    Google Scholar 

  16. Mehrotra K G, Mohan C K, Huang H (2017) Anomaly detection principles and algorithms. Springer

  17. Park JW, Kim DY (2017) Standard time estimation of manual tasks via similarity measure of unequal scale time series. IEEE Transactions on Human-Machine Systems 48(3):241–251

    Article  Google Scholar 

  18. Ren H, Ye Z, Li Z (2017) Anomaly detection based on a dynamic Markov model. Inf Sci 411:52–65

    Article  MathSciNet  Google Scholar 

  19. Li J, Pedrycz W, Jamal I (2017) Multivariate time series anomaly detection: a framework of hidden Markov models. Appl Soft Comput 60:229–240

    Article  Google Scholar 

  20. F-h S, Sun D (2010) PLR based on time series tendency turning point. Application Research of Computers 6(27):2075–2074

    Google Scholar 

  21. Nakamura T, Taki K, Nomiya H (2013) A shape-based similarity measure for time series data with ensemble learning. Pattern Anal Applic 16(4):535–548

    Article  MathSciNet  Google Scholar 

  22. Kolozali Ş, Puschmann D, Bermudez-Edo M (2016) On the effect of adaptive and nonadaptive analysis of time-series sensory data. IEEE Internet Things J 3(6):1084–1098

    Article  Google Scholar 

  23. Dwivedi Y, Subba Rao S (2011) A test for second-order stationarity of a time series based on the discrete Fourier transform. J Time Ser Anal 32(1):68–91

    Article  MathSciNet  Google Scholar 

  24. Chaovalit P, Gangopadhyay A (2011) Discrete wavelet transform-based time series analysis and mining. ACM Computing Surveys (CSUR) 43(2):1–37

    Article  Google Scholar 

  25. Yazdi SV, Douzal-Chouakria A (2018) Time warp invariant kSVD: sparse coding and dictionary learning for time series under time warp. Pattern Recogn Lett 112:1–8

    Article  Google Scholar 

  26. Cai Q, Chen L, Sun J (2015) Piecewise statistic approximation based similarity measure for time series. Knowl-Based Syst 85:181–195

    Article  Google Scholar 

  27. Keogh E, Wei L, Xi X (2009) Supporting exact indexing of arbitrarily rotated shapes and periodic time series under euclidean and warping distance measures. VLDB J 18(3):611–630

    Article  Google Scholar 

  28. Rakthanmanon T, Campana B, Mueen A (2013) Addressing big data time series: mining trillions of time series subsequences under dynamic time warping. ACM Transactions on Knowledge Discovery from Data (TKDD) 7(3):1–31

    Article  Google Scholar 

  29. Fang L, Wan M, Yu M (2015) Analysis of similarity measure in the longitudinal study using improved longest common subsequence method for lung cancer. Biomedical Signal Processing and Control 15:60–66

    Article  Google Scholar 

  30. Ren H, Li X, Li Z, Pedrycz W (2018) Data representation based on interval-sets for anomaly detection in time series. IEEE Access 6(99):27473–27479

    Article  Google Scholar 

  31. Izakian H, Pedrycz W, Jamal I (2015) Fuzzy clustering of time series data using dynamic time warping distance. Eng Appl Artif Intell 39:235–244

    Article  Google Scholar 

  32. Münz G, Li S, Carle G (2007) Traffic anomaly detection using k-means clustering. In proceedings of the GI/ITG workshop MMBnet: 13-14

  33. Huang H, Mehrotra K, Mohan C K (2012) Algorithms for detecting outliers via clustering and ranks. In Proceedings of the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: 20–29

  34. Van Der Voort M, Dougherty M, Watson S (1996) Combining Kohonen maps with ARIMA time series models to forecast traffic flow. Transportation Research Part C: Emerging Technologies 4(5):307–318

    Article  Google Scholar 

  35. Karim F, Majumdar S, Darabi H (2017) LSTM fully convolutional networks for time series classification. IEEE Access 6(99):1662–1669

    Google Scholar 

  36. Pedrycz W, Lu W, Liu X (2014) Human-centric analysis and interpretation of time series: a perspective of granular computing. Soft Comput 18(12):2397–2411

    Article  Google Scholar 

  37. Froelich W, Pedrycz W (2017) Fuzzy cognitive maps in the modeling of granular time series. Knowl-Based Syst 115:110–122

    Article  Google Scholar 

  38. Su M-Y (2011) Real-time anomaly detection systems for denial-of-service attacks by weighted k-nearest-neighbor classifiers. Expert Syst Appl 38(4):3492–3498

    Article  Google Scholar 

  39. Yahyaoui H, Al-Daihani R (2019) A novel trend based SAX reduction technique for time series. Expert Syst Appl 130:113–123

    Article  Google Scholar 

  40. Yan Q-Y, Xia S-X, Feng K-W (2012) Probabilistic distance based abnormal pattern detection in uncertain series data. Knowl-Based Syst 36:182–190

    Article  Google Scholar 

  41. Sternickel K (2002) Automatic pattern recognition in ECG time series. Comput Methods Prog Biomed 68(2):109–115

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the Raytheon Chair for Systems Engineering for funding. This work was supported in part by the Fundamental Research Funds for the Central Universities under Grant K50510040013 and Grant K5051304007, in part by the Natural Science Foundation of China under Grant 61374068, and in part by the Science and Technology Development Fund, MSAR, under Grant 078/2015/A3, and the Doctoral Students’ Short-Term Study Abroad Scholarship Fund of Xidian University.

Author information

Authors and Affiliations

  1. School of Electro-Mechanical Engineering, Xidian University, Xi’an, 710071, China

    Yanjun Zhou, Huorong Ren & Zhiwu Li

  2. Institute of Systems Engineering, Macau University of Science and Technology, Macau, 999078, Macao, Special Administrative Region of China

    Zhiwu Li & Naiqi Wu

  3. National Key Laboratory of Precise Electronic Manufacturing Technology and Equipment, Guangdong University of Technology, Guangzhou, 510006, China

    Naiqi Wu

  4. Raytheon Chair for Systems Engineering (RCSE Chair), Advanced Manufacturing Institute, King Saud University, Riyadh, 11421, Saudi Arabia

    Abdulrahman M. Al-Ahmari

Authors
  1. Yanjun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huorong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiwu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naiqi Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdulrahman M. Al-Ahmari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huorong Ren.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Ren, H., Li, Z. et al. Anomaly detection via a combination model in time series data. Appl Intell 51, 4874–4887 (2021). https://doi.org/10.1007/s10489-020-02041-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02041-3

Keywords

Search

Navigation