Skip to main content
Springer Nature Link
Log in

Optimal Parameter Selection Using Explainable AI for Time-Series Anomaly Detection

  • Conference paper
  • First Online:
PRIMA 2022: Principles and Practice of Multi-Agent Systems (PRIMA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13753))

  • 1105 Accesses

Abstract

Time-series anomaly detection is a technique for detecting unusual values, changes, or movements in a large amount of data arranged in time-series. It is primarily used in the fields of intrusion detection, medical diagnosis, and industrial defect damage detection and necessary to realize agents that operate intelligently and autonomously, such as changing system behavior based on detected anomalies. SALAD is a real-time time-series anomaly detection method based on deep learning. It is lightweight and determines anomaly detection threshold flexibly; however, experts need to determine an appropriate value for a parameter so that it suits any given recurrent time series, and this inhibits the realization of the agent. In this study, we propose a method to determine automatically the optimal parameter value in SALAD’s prediction model by utilizing XAI. We use SHAP, which provides interpretability to the prediction by the deep learning model. Through evaluation experiment, we demonstrate that our method is effective and provide an example of the use of XAI for time-series anomaly detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Schölkopf, B., Platt, J.C., Shawe-Taylor, J.C., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)

    Article  MATH  Google Scholar 

  2. Hawkins, D.: Identification of outliers. Chapman and hall, London (1980)

    Book  MATH  Google Scholar 

  3. Chalapathy, R., Chawla, S.: Deep learning for anomaly detection: a survey. arXiv preprint arXiv:1901.03407 (2019)

  4. Hayes, M.A., Capretz, M.A.M.: Contextual anomaly detection framework for big sensor data. J. Big Data 2(1), 1–22 (2015). https://doi.org/10.1186/s40537-014-0011-y

    Article  Google Scholar 

  5. Lee, M.-C., Lin, J.-C., Gran, E.G.: SALAD: self-adaptive lightweight anomaly detection for real-time recurrent time series. In: Proceedings of the 45th IEEE Computer Society Signature Conference on Computers, Software, and Applications (COMPSAC 2021), pp. 344–349 (2021). arXiv preprint arXiv:2104.09968

  6. Hochenbaum, J., Vallis, O.S., Kejariwal, A.: Automatic anomaly detection in the cloud via statistical learning. arXiv preprint arXiv:1704.07706 (2017)

  7. Pukelsheim, F.: The three sigma rule. Am. Stat. 48(2), 88–91 (1994)

    MathSciNet  Google Scholar 

  8. Senin, P., et al.: GrammarViz 3.0: interactive discovery of variable-length time series patterns. ACM Trans. Knowl. Discov. Data (TKDD) 12(1), 1–28 (2018)

    Article  Google Scholar 

  9. linkedin/luminol. https://github.com/linkedin/luminol

  10. Laptev, N., Amizadeh, S., Flint, I.: Generic and scalable framework for automated time-series anomaly detection. In: Proceedings of the 21th ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 1939–1947 (2015)

    Google Scholar 

  11. lavin, A., Ahmad, S.: Evaluating real-time anomaly detection algorithms-the numenta anomaly benchmark. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA 2015), pp. 38–44 (2015)

    Google Scholar 

  12. Zhang, W., et al.: LSTM-based analysis of industrial IoT equipment. IEEE Access 6, 23551–23560 (2018)

    Article  Google Scholar 

  13. Mudassar, B.A., Ko, J.H., Mukhopadhyay, S.: An unsupervised anomalous event detection frame-work with class aware source separation. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2671–2675 (2018)

    Google Scholar 

  14. Schmidt-Erfurth, U., Sadeghipour, A., Gerendas, B.S., Waldstein, M., Bogunovic, H.: Artificial intelligence in retina. Prog. Retinal Eye Res. 67, 1–29 (2018)

    Article  Google Scholar 

  15. Xu, H., et al.: Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 187–196 (2018)

    Google Scholar 

  16. Le, Q., Namee, B.M., Scanlon, M.: Deep learning at the shallow end: malware classification for non-domain experts. Digit. Investig. 26, S118–S126 (2018)

    Article  Google Scholar 

  17. HaddadPajouh, H., Dehghantanha, A., Khayami, R., Choo, K.-K.R.: A deep recurrent neural network based approach for internet of things malware threat hunting. Future Gener. Comput. Syst. 85, 88–96 (2018)

    Article  Google Scholar 

  18. Kanarachos, S., Christopoulos, S.-R.G., Chroneos, A., Fitzpatrick, M.E.: Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and Hilbert transform. Expert Syst. Appl. 85, 292–304 (2017)

    Article  Google Scholar 

  19. Shipmon, D., Gurevitch, J., Piselli, P.M., Edwards, S.: Time series anomaly detection: detection of anomalous drops with limited features and sparse examples in noisy periodic data. Technical report, Google Inc. (2017). https://arxiv.org/abs/1708.03665

  20. Lee, M.-C., Lin, J.-C., Gran, E.G.: RePAD: real-time proactive anomaly detection for time series. In: Barolli, L., Amato, F., Moscato, F., Enokido, T., Takizawa, M. (eds.) AINA 2020. AISC, vol. 1151, pp. 1291–1302. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-44041-1_110

    Chapter  Google Scholar 

  21. Lee, M-C., Lin, J-C., Gran, E.G.: ReRe: a lightweight real- time ready-to-go anomaly detection approach for time series.: In: Proceedings of the 44th IEEE Computer Society Signature Conference on Computers, Software, and Applications (COMPSAC 2020), pp. 322–327 (2020)

    Google Scholar 

  22. Lee, M.-C., Lin, J.-C., Gran, E.G.: Distributed fine-grained traffic speed prediction for large-scale transportation networks based on automatic LSTM customization and sharing. In: Malawski, M., Rzadca, K. (eds.) Euro-Par 2020. LNCS, vol. 12247, pp. 234–247. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57675-2_15

    Chapter  Google Scholar 

  23. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 1135–1144 (2016)

    Google Scholar 

  24. Lundberg S., Lee, S.-I.: A unified approach to interpreting model predictions. In: NIPS 2017: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 4768–4777 (2017)

    Google Scholar 

  25. Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189–1232 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  26. Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E.: Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J. Comput. Graph. Stat. 24(1), 44–65 (2015)

    Article  MathSciNet  Google Scholar 

  27. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization (2016)

    Google Scholar 

  28. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 618–626 (2017)

    Google Scholar 

  29. Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. arXiv:1801.01489 (2018)

  30. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML 2017: Proceedings of the 34th International Conference on Machine Learning, Vol. 70, pp, 3319–3328 (2017)

    Google Scholar 

  31. Diallo, A.B., Nakagawa, H., Tsuchiya, T.: Adaptation space reduction using an explainable framework. In: Proceedings of the IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC), pp. 1653–1660 (2021)

    Google Scholar 

  32. numenta/NAB. https://github.com/numenta/NAB

  33. A tour of AI technologies in time series prediction. https://www.soa.org/resources/research-reports/2019/tourai-technologies/

  34. Farahani, I.V., Chien, A., King, R.E., Kay, M.G., Klenz, B.: Time series anomaly detection from a Markov chain perspective. In: 2019 IEEE 18th International Conference on Machine Learning and Applications (ICMLA), pp. 1000–1007 (2019)

    Google Scholar 

Download references

Acknowledgment

This work was supported by JSPS Grants-in-Aid for Scientific Research (Grant Numbers 17KT0043, 20H04167, and 18H03229).

Author information

Authors and Affiliations

  1. Osaka University, Osaka, Japan

    Shimon Sumita, Hiroyuki Nakagawa & Tatsuhiro Tsuchiya

Authors
  1. Shimon Sumita
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hiroyuki Nakagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tatsuhiro Tsuchiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shimon Sumita .

Editor information

Editors and Affiliations

  1. Özyeğin University, Istanbul, Turkey

    Reyhan Aydoğan

  2. Universitat Politècnica de València, Valencia, Spain

    Natalia Criado

  3. Université Paris-Dauphine, Paris, France

    Jérôme Lang

  4. Universitat Politècnica de València, Valencia, Spain

    Victor Sanchez-Anguix

  5. King's College London, London, UK

    Marc Serramia

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sumita, S., Nakagawa, H., Tsuchiya, T. (2023). Optimal Parameter Selection Using Explainable AI for Time-Series Anomaly Detection. In: Aydoğan, R., Criado, N., Lang, J., Sanchez-Anguix, V., Serramia, M. (eds) PRIMA 2022: Principles and Practice of Multi-Agent Systems. PRIMA 2022. Lecture Notes in Computer Science(), vol 13753. Springer, Cham. https://doi.org/10.1007/978-3-031-21203-1_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21203-1_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21202-4

  • Online ISBN: 978-3-031-21203-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics