Skip to main content
SpringerLink
Log in

Attention-Based Bi-LSTM for Anomaly Detection on Time-Series Data

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2021 (ICANN 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12891))

Included in the following conference series:

Abstract

Anomaly detection in time-series data is a significant research problem that has applications in multiple areas. Unsupervised anomaly detection is a fundamental aspect of developing intelligent automated systems. Existing work in this field has primarily focused on developing intelligent systems that use dimensionality reduction or regression-based approaches to annotate data based on a certain static threshold. Researchers in fields such as Natural Language Processing (NLP) and Computer Vision (CV) have realized considerable improvement by incorporating attention in prediction-related tasks. In this work, we propose an attention-based bi-directional long short term memory (Attention-Bi-LSTM) networks for anomaly detection on time-series data. It helps in assigning optimal weights to instances in sequential data. We evaluate the proposed approach on the entirety of the popularly used Numenta Anomaly Benchmark (NAB). Additionally, we also contribute by creating new baselines on the NAB with recent models such as REBM, DAGMM, LSTM-ED, and Donut, which have not been previously used on the NAB.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    All code and models to reproduce the results of this paper are available in this Github repository: https://github.com/Varad2305/Time-Series-Anomaly-Detection.

  2. 2.

    Similar results are obtained for other datasets not depicted in this work. In this paper, visualizations are restricted to the realTweets and realTraffic datasets only.

  3. 3.

    We use the four recent models from the KDD-OpenSource Repository DeepADoTSon Github - DAGMM (2018), Donut (2018) REBM (2016) and LSTMED (2016).

References

  1. Ahmad, S., Purdy, S.: Real-time anomaly detection for streaming analytics. arXiv preprint arXiv:1607.02480 (2016)

  2. Ahmad, S., Lavin, A., Purdy, S., Agha, Z.: Unsupervised real-time anomaly detection for streaming data. Neurocomputing 262, 134–147 (2017). https://doi.org/10.1016/j.neucom.2017.04.070. ISSN 0925–2312

    Article  Google Scholar 

  3. Baziotis, C., Pelekis, N., Doulkeridis, C.: DataStories at SemEval-2017 task 4: deep LSTM with attention for message-level and topic-based sentiment analysis. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval 2017), Vancouver, Canada, August 2017, pp. 747–754. https://doi.org/10.18653/v1/S17-2126

  4. Bergstra, J., Yamins, D., Cox, D.: Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: International Conference on Machine Learning, pp. 115–123. PMLR (2013)

    Google Scholar 

  5. Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: 25th Annual Conference on Neural Information Processing Systems (NIPS 2011), vol. 24. Neural Information Processing Systems Foundation (2011)

    Google Scholar 

  6. Chalapathy, R., Chawla, S.: Deep learning for anomaly detection: a survey. arXiv preprint arXiv:1901.03407 (2019)

  7. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)

    Article  Google Scholar 

  8. Etsy: etsy/skyline. https://github.com/etsy/skyline

  9. Fukui, H., Hirakawa, T., Yamashita, T., Fujiyoshi, H.: Attention branch network: Learning of attention mechanism for visual explanation. CoRR, abs/1812.10025 (2018). http://arxiv.org/abs/1812.10025

  10. Hu, D.: An introductory survey on attention mechanisms in NLP problems. In: Bi, Y., Bhatia, R., Kapoor, S. (eds.) SAI Intelligent Systems Conference. AISC, vol. 1037, pp. 432–448. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29516-5

    Chapter  Google Scholar 

  11. Huang, C., Min, G., Wu, Y., Ying, Y., Pei, K., Xiang, Z.: Time series anomaly detection for trustworthy services in cloud computing systems. IEEE Trans. Big Data, 1 (2017). https://doi.org/10.1109/TBDATA.2017.2711039

  12. Introducing practical and robust anomaly detection in a time series, January 2015. https://blog.twitter.com/engineering/en_us/a/2015/introducing-practical-and-robust-anomaly-detection-in-a-time-series.html

  13. Karim, F., Majumdar, S., Darabi, H., Chen, S.: LSTM fully convolutional networks for time series classification. IEEE Access 6, 1662–1669 (2017)

    Article  Google Scholar 

  14. Lavin, A., Ahmad, S.: Evaluating real-time anomaly detection algorithms-the numenta anomaly benchmark. In: 2015 IEEE 14th International Conference on Machine Learning and Applications, pp. 38–44. IEEE (2015)

    Google Scholar 

  15. Lee, I., Lee, K.: The internet of things (IoT): applications, investments, and challenges for enterprises. Bus. Horiz. 58(4), 431–440 (2015)

    Article  Google Scholar 

  16. Lomb, N.R.: Least-squares frequency analysis of unequally spaced data. Astrophys. Space Sci. 39(2), 447–462 (1976)

    Article  Google Scholar 

  17. Malhotra, P., Ramakrishnan, A., Anand, G., Vig, L., Agarwal, P., Shroff, G.: LSTM-based encoder-decoder for multi-sensor anomaly detection, July 2016

    Google Scholar 

  18. Munir, M., Siddiqui, S.A., Dengel, A., Ahmed, S.: DeepAnT: a deep learning approach for unsupervised anomaly detection in time series. IEEE Access 7, 1991–2005 (2019). https://doi.org/10.1109/ACCESS.2018.2886457

    Article  Google Scholar 

  19. Munir, M., Siddiqui, S., Chattha, M., Dengel, A., Ahmed, S.: FuseAD: unsupervised anomaly detection in streaming sensors data by fusing statistical and deep learning models. Sensors 19, 05 (2019). https://doi.org/10.3390/s19112451

    Article  Google Scholar 

  20. Pereira, J., Silveira, M.: Unsupervised anomaly detection in energy time series data using variational recurrent autoencoders with attention. In: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1275–1282 (2018). https://doi.org/10.1109/ICMLA.2018.00207

  21. Sadouk, L.: CNN approaches for time series classification. In: Time Series Analysis-Data, Methods, and Applications, pp. 1–23. IntechOpen (2019)

    Google Scholar 

  22. Scargle, J.D.: Studies in astronomical time series analysis. II-statistical aspects of spectral analysis of unevenly spaced data. Astrophys. J. 263, 835–853 (1982)

    Article  Google Scholar 

  23. Vaswani, A., et al.: Attention is all you need (2017). https://arxiv.org/pdf/1706.03762.pdf

  24. Xu, H., et al.: Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications. In: Proceedings of the 2018 World Wide Web Conference, pp. 187–196 (2018)

    Google Scholar 

  25. Yoon, C., Huh, M., Kang, S.-G., Park, J., Lee, C.: Implement smart farm with IoT technology. In: 2018 20th International Conference on Advanced Communication Technology (ICACT), pp. 749–752. IEEE (2018)

    Google Scholar 

  26. Yu, Y., Liu, G., Yan, H., Li, H., Guan, H.: Attention-based BI-LSTM model for anomalous http traffic detection. In: 2018 15th International Conference on Service Systems and Service Management (ICSSSM), pp. 1–6 (2018). https://doi.org/10.1109/ICSSSM.2018.8465034

  27. Zhai, S., Cheng, Y., Lu, W., Zhang, Z.: Deep structured energy based models for anomaly detection. arXiv preprint arXiv:1605.07717 (2016)

  28. Zhao, H.G.: Keras-self-attention (2018). https://github.com/CyberZHG

  29. Zheng, G., Mukherjee, S., Dong, X.L., Li, F.: OpenTag: open attribute value extraction from product profiles. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1049–1058. Association for Computing Machinery (2018). https://doi.org/10.1145/3219819.3219839. ISBN 9781450355520

  30. Zong, B., et al.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: International Conference on Learning Representations (2018)

    Google Scholar 

Download references

Acknowledgement

The authors would like to thank TCS R&D for funding this research through PhD fellowship to the first author.

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Systems, BITS Pilani - Hyderabad Campus, Hyderabad, Telangana, India

    Sanket Mishra, Varad Kshirsagar, Rohit Dwivedula & Chittaranjan Hota

  2. School of Computer Science and Engineering, Vellore Institute of Technology, Amaravati, Andhra Pradesh, India

    Sanket Mishra

Authors
  1. Sanket Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Varad Kshirsagar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rohit Dwivedula
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chittaranjan Hota
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanket Mishra .

Editor information

Editors and Affiliations

  1. Comenius University in Bratislava, Bratislava, Slovakia

    Igor Farkaš

  2. iMotions A/S, Copenhagen, Denmark

    Paolo Masulli

  3. University of Tübingen, Tübingen, Baden-Württemberg, Germany

    Sebastian Otte

  4. Universität Hamburg, Hamburg, Germany

    Stefan Wermter

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mishra, S., Kshirsagar, V., Dwivedula, R., Hota, C. (2021). Attention-Based Bi-LSTM for Anomaly Detection on Time-Series Data. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12891. Springer, Cham. https://doi.org/10.1007/978-3-030-86362-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86362-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86361-6

  • Online ISBN: 978-3-030-86362-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation