Skip to main content
SpringerLink
Log in

A novel multi-level framework for anomaly detection in time series data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Anomaly detection is a challenging problem in science and engineering that appeals to numerous scholars. It is of great relevance to detect anomalies and analyze their potential implications. In this study, a multi-level anomaly detection framework with information granules of higher type and higher order is developed based on the principle of justifiable granularity and Fuzzy C-Means (FCM) clustering algorithm, including two different types of approaches, namely abstract level approach (ALA) and detailed level approach (DLA). The ALA approach is implemented at a comparatively abstract level (viz., level-1), in which two distinct types of information granules of order-1 (viz., information granules of type-1 and type-2) are employed for anomaly detection. The DLA approach is formulated and derived from the ALA approach at a more detailed level (viz., level-2), which generates more detailed information granules, namely information granules of order-2, through successive splitting information granules and the FCM clustering algorithm to refine the problem at various levels. Furthermore, a similarity measurement algorithm is designed for anomaly detection utilizing information granules of higher type and higher order. Comprehensive performance indexes are produced to quantify the performance of the proposed framework compared with the methods of two single-level approaches and two multi-level approaches. Synthetic data and several real-world data coming from various areas are engaged to demonstrate and support the superiority of the proposed approaches over other classical methods in terms of detection accuracy and data anomaly resolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Zhao F, Zeng G, Lu K (2019) EnLSTM-WPEO: short-term traffic flow prediction by ensemble LSTM , NNCT weight integration, and population extremal optimization. IEEE Trans Veh Technol 69(1):101–113

    Article  Google Scholar 

  2. Chen M, Zeng G, Lu K (2019) A two-layer nonlinear combination method for short-term wind speed prediction based on ELM, ENN, and LSTM. IEEE Internet Things J 6(4):6997–7010

    Article  Google Scholar 

  3. Li H, Boulanger P (2020) A survey of heart anomaly detection using ambulatory electrocardiogram (ECG). Sensors 20(5):1461

    Article  Google Scholar 

  4. Kumar S, Toshniwal D (2016) A novel framework to analyze road accident time series data. J Big Data 3(1):1–11

    Article  Google Scholar 

  5. Zhou Y, Ren H, Li Z, Pedrycz W (2021) Anomaly detection via a combination model in time series data. Appl Intell 51(7):4874–4887

    Article  Google Scholar 

  6. Li J, Pedrycz W, Jamal I (2017) Multivariate time series anomaly detection: a framework of hidden Markov models. Appl Soft Comput 60:229–240

    Article  Google Scholar 

  7. Habeeb RAA, Nasaruddin F, Gani A (2019) Real-time big data processing for anomaly detection: a survey. Int J Inf Manag 45:289–307

    Article  Google Scholar 

  8. Blzquez-Garca A, Conde A, Mori U (2021) A review on outlier/anomaly detection in time series data. ACM Comput Surveys (CSUR) 54(3):1–33

    Article  Google Scholar 

  9. Mishra K, Basu S, Maulik U (2022) Graft: a graph based time series data mining framework. Eng Appl Artif Intell 110:104695

    Article  Google Scholar 

  10. Zhou Y, Ren H, Li Z, Pedrycz W (2021) An anomaly detection framework for time series data: an interval-based approach. Knowl-Based Syst 228:107153

    Article  Google Scholar 

  11. Moeyersons J, Smets E, Morales J (2019) Artefact detection and quality assessment of ambulatory ECG signals. Comput Methods Programs Biomed 182:105050

    Article  Google Scholar 

  12. Izakian H, Pedrycz W (2013) Anomaly detection in time series data using a fuzzy c-means clustering. 2013 Joint IFSA world congress and NAFIPS annual meeting (IFSA/NAFIPS) pp 1513–1518

  13. Zhu X, Pedrycz W, Li Z (2016) Granular encoders and decoders: a study in processing information granules. IEEE Trans Fuzzy Syst 25(5):1115–1126

    Article  Google Scholar 

  14. Pedrycz W, Lu W, Liu X (2014) Human-centric analysis and interpretation of time series: a perspective of granular computing. Soft Comput 18(12):2397–2411

    Article  Google Scholar 

  15. Ren H, Liu M, Liao X (2018) Anomaly detection in time series based on interval sets. IEEJ Trans Electr Electron Eng 13(5):757–762

    Article  Google Scholar 

  16. Ren H, Liu M, Li Z, Pedrycz W (2017) A piecewise aggregate pattern representation approach for anomaly detection in time series. Knowl-Based Syst 135:29–39

    Article  Google Scholar 

  17. Jiang W, Hong Y, Zhou B (2019) A GAN-based anomaly detection approach for imbalanced industrial time series. IEEE Access 7:143608–143619

    Article  Google Scholar 

  18. Ren H, Ye Z, Li Z (2017) Anomaly detection based on a dynamic Markov model. Inf Sci 411:52–65

    Article  MathSciNet  Google Scholar 

  19. He H, Tan Y (2017) Automatic pattern recognition of ECG signals using entropy-based adaptive dimensionality reduction and clustering. Appl Soft Comput 55:238–252

    Article  Google Scholar 

  20. Howedi A, Lotfi A, Pourabdollah A (2020) A multi-scale fuzzy entropy measure for anomaly detection in activities of daily living. In: Proceedings of the 13th ACM international conference on PErvasive technologies related to assistive environments, pp 1–8

  21. Huang L, Ran J, Wang W (2021) A multi-channel anomaly detection method with feature selection and multi-scale analysis. Comput Netw 185:107645

    Article  Google Scholar 

  22. Chen X, Zhan Y (2008) Multi-scale anomaly detection algorithm based on infrequent pattern of time series. J Comput Appl Math 214(1):227–237

    Article  MathSciNet  MATH  Google Scholar 

  23. Zhu X, Pedrycz W, Li Z (2019) Development and analysis of neural networks realized in the presence of granular data. IEEE Trans Neural Netw Learn Syst 31(9):3606–3619

    Article  Google Scholar 

  24. Zhang B, Pedrycz W, Wang X (2020) Design of interval type-2 information granules based on the principle of justifiable granularity. IEEE Trans Fuzzy Syst

  25. Pedrycz W, Al-Hmouz R, Balamash AS (2015) Hierarchical granular clustering: an emergence of information granules of higher type and higher order. IEEE Trans Fuzzy Syst 23(6):2270–2283

    Article  Google Scholar 

  26. Gacek A, Pedrycz W (2014) Clustering granular data and their characterization with information granules of higher type. IEEE Trans Fuzzy Syst 23(4):850–860

    Article  Google Scholar 

  27. Liu S, Pedrycz W, Gacek A (2018) Development of information granules of higher type and their applications to granular models of time series. Eng Appl Artif Intell 71:60–72

    Article  Google Scholar 

  28. Pedrycz W, Bargiela A (2011) An optimization of allocation of information granularity in the interpretation of data structures: toward granular fuzzy clustering. IEEE Trans Syst Man, and Cybern 42(3):582–590

    Article  Google Scholar 

  29. Izakian H, Pedrycz W, Jamal I (2012) Clustering spatiotemporal data: an augmented fuzzy C-means. IEEE Trans Fuzzy Syst 21(5):855–868

    Article  Google Scholar 

  30. Pedrycz W, Wang X (2015) Designing fuzzy sets with the use of the parametric principle of justifiable granularity. IEEE Trans Fuzzy Syst 24(2):489–496

    Article  Google Scholar 

  31. Pedrycz W, Homenda W (2013) Building the fundamentals of granular computing: a principle of justifiable granularity. Appl Soft Comput 13(10):4209–4218

    Article  Google Scholar 

  32. Wang D, Pedrycz W, Li Z (2018) Granular data aggregation: an adaptive principle of the justifiable granularity approach. IEEE Trans Cybern 49(2):417–426

    Article  Google Scholar 

  33. Lu W, Chen X, Pedrycz W (2015) Using interval information granules to improve forecasting in fuzzy time series. Int J Approx Reason 57:1–18

    Article  MATH  Google Scholar 

  34. Li J, Izakian H, Pedrycz W (2021) Clustering-based anomaly detection in multivariate time series data. Appl Soft Comput 100:106919

    Article  Google Scholar 

  35. Keogh E, Lin J, Fu A (2005) Hot SAX: efficiently finding the most unusual time series subsequence. Fifth IEEE Int Conf Data Mining (ICDM’05)

Download references

Acknowledgements

This work was supported in part by the Fundamental Research Funds for the Central Universities under Grant K50510040013 and Grant K5051304007, in part by the Natural Science Foundation of China under Grant 61374068, and in part by the Science and Technology Development Fund, MSAR, under Grant 078/2015/A3, and the Doctoral Students Study Abroad Scholarship Fund of Xidian University.

Author information

Authors and Affiliations

  1. School of Electro-Mechanical Engineering, Xidian University, Xi’an, 710071, China

    Yanjun Zhou, Huorong Ren, Dan Zhao & Zhiwu Li

  2. Institute of Systems Engineering, Macau University of Science and Technology, Taipa Macau, SAR 999078, China

    Zhiwu Li

  3. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, T6G 2V4, Canada

    Witold Pedrycz

Authors
  1. Yanjun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huorong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiwu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Witold Pedrycz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huorong Ren.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Ren, H., Zhao, D. et al. A novel multi-level framework for anomaly detection in time series data. Appl Intell 53, 10009–10026 (2023). https://doi.org/10.1007/s10489-022-04016-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04016-y

Keywords

Search

Navigation