skip to main content
10.1145/2513456.2513490acmotherconferencesArticle/Chapter ViewAbstractPublication PageshtConference Proceedingsconference-collections
research-article

Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data

Published: 07 October 2013 Publication History

Abstract

This paper evaluates the performance of long short-term memory recurrent neural networks (LSTM-RNN) on classifying intrusion detection data. LSTM networks can learn memory and can therefore model data as a time series. LSTM is trained and tested on a processed version of the KDDCup99 dataset. A variety of suitable performance measures are discussed and applied. Our LSTM network structure and parameters are experimentally obtained within a series of experiments presented. Results finally show that LSTM is able to learn all attack classes hidden in the training data. Furthermore we learn that the receiver operating characteristic (ROC) curve and the corresponding area-under-the-curve (AUC) value are well suited for selecting well performing networks.

References

[1]
S. Chavan, K. Shah, N. Dave, S. Mukherjee, A. Abraham, and S. Sanyal. Adaptive neuro-fuzzy intrusion detection systems. In International Conference on Information Technology: Coding and Computing, volume 1, pages 70--74 Vol.1. IEEE, 2004.
[2]
E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson. Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics, 44(3):837, Sept. 1988.
[3]
C. Elkan. Results of the KDD'99 classifier learning. ACM SIGKDD Explorations Newsletter, 1(2):63, Jan. 2000.
[4]
J. L. Elman. Finding Structure in Time. Cognitive Science, 14(2):179--211, Mar. 1990.
[5]
T. Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, 27(8):861--874, June 2006.
[6]
F. A. F. A. Gers, J. J. Schmidhuber, and F. Cummins. Learning to forget: Continual prediction with LSTM. Technical Report IDSIA-01-99, IDSIA, Lugano, Lugano, CH, Oct. 1999.
[7]
S. Hochreiter and J. Schmidhuber. Long short-term memory. Technical Report 8, Technische Universität Muenchen, 1997.
[8]
M. I. Jordan. Attractor dynamics and parallelism in a connectionist sequentialmachine. In Proceedings of the Eigth Annual Conference of the Cognitive Science Society, pages 531--546, 1986.
[9]
W. Lee and S. J. Stolfo. A framework for constructing features and models for intrusion detection systems. ACM Transactions on Information and System Security, 3(4):227--261, Nov. 2000.
[10]
R. Lippmann, J. W. Haines, D. J. Fried, J. Korba, and K. Das. The 1999 DARPA off-line intrusion detection evaluation. Computer Networks, 34(4):579--595, Oct. 2000.
[11]
R. P. Lippmann, D. J. Fried, I. Graf, J. W. Haines, K. R. Kendall, D. McClung, D. Weber, S. E. Webster, D. Wyschogrod, R. K. Cunningham, M. Zissman, and Others. Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation. In Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00, volume 2, pages 12--26. IEEE, IEEE Comput. Soc, 2000.
[12]
M. V. Mahoney and P. K. Chan. An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection. Recent Advances in Intrusion Detection, 2820(Ll):220--237, 2003.
[13]
M. A. Maloof. Some basic concepts of machine learning and data mining. In Machine Learning and Data Mining for Computer Security, Advanced Information and Knowledge Processing, pages 23--43. Springer London, 2006.
[14]
J. McHugh. Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory. ACM Transactions on Information and System Security, 3(4):262--294, Nov. 2000.
[15]
C. E. Metz, Y. Jiang, H. MacMahon, R. M. Nishikawa, and X. Ran. ROC software. http://metz-roc.uchicago.edu/. Accessed: 2013-08-27.
[16]
S. Mukkamala, G. Janoski, and A. Sung. Intrusion detection using neural networks and support vector machines. In Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290), pages 1702--1707. IEEE, 2002.
[17]
S. Peddabachigari, A. Abraham, C. Grosan, and J. Thomas. Modeling intrusion detection system using hybrid intelligent systems. Journal of Network and Computer Applications, 30(1):114--132, Jan. 2007.
[18]
L. L. Pesce and C. E. Metz. Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. Academic radiology, 14(7):814--29, July 2007.
[19]
J. R. J. Quinlan. C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., Mar. 1993.
[20]
D. Z. Ronald J. Williams, R. J. R. Williams, and D. Zipser. Gradient-based learning algorithms for recurrent networks and their computational complexity. In Back-propagation: Theory, Architectures and Applications, pages 1--45. L. Erlbaum Associates Inc., Jan. 1995.
[21]
D. E. Rumelhart, G. E. Hinton, R. J. Williams, D. Rummelhart, and W. R. J. Learning Internal Representations by Error Propagation. In J. L. McClelland and D. E. Rumelhart, editors, Parallel distributed processing: explorations in the microstructure of cognition, volume 1, pages 318--362. MIT Press, Jan. 1986.
[22]
M. Sabhnani and G. Serpen. Application of machine learning algorithms to KDD intrusion detection dataset within misuse detection context. In Proceedings of the International Conference on Machine Learning; Models, Technologies and Applications, pages 209--215. CSREA Press, 2003.
[23]
M. Sabhnani and G. Serpen. Why machine learning algorithms fail in misuse detection on KDD intrusion detection data set. Intelligent Data Analysis, 6(2002):1--13, Sept. 2004.
[24]
M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani. A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, number Cisda, pages 1--6. IEEE, July 2009.
[25]
P. Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10):1550--1560, 1990.
[26]
R. J. Williams and D. Zipser. Experimental Analysis of the Real-time Recurrent Learning Algorithm. Connection Science, 1(1):87--111, Jan. 1989.

Cited By

View all
  • (2025)An optimized LSTM-based deep learning model for anomaly network intrusion detectionScientific Reports10.1038/s41598-025-85248-z15:1Online publication date: 10-Jan-2025
  • (2024)A Comparative Analysis of Convolutional, Sequential and Their Hybrid Models in Detecting Cyber-Attacks2024 1st International Conference on Trends in Engineering Systems and Technologies (ICTEST)10.1109/ICTEST60614.2024.10576092(01-08)Online publication date: 11-Apr-2024
  • (2024)Unveiling the Landscape of Machine Learning and Deep Learning Methodologies in Network Security: A Comprehensive Literature Review2024 2nd International Conference on Cyber Resilience (ICCR)10.1109/ICCR61006.2024.10533066(1-7)Online publication date: 26-Feb-2024
  • Show More Cited By
  1. Evaluating performance of long short-term memory recurrent neural networks on intrusion detection data

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      SAICSIT '13: Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference
      October 2013
      398 pages
      ISBN:9781450321129
      DOI:10.1145/2513456
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      • Amazon: Amazon.com
      • Rhodes Univ.: Rhodes University
      • IBM: IBM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 October 2013

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. KDDCup99
      2. intrusion detection systems
      3. long short-term memory
      4. machine learning
      5. receiver operating characteristic
      6. recurrent neural networks
      7. time series analysis

      Qualifiers

      • Research-article

      Conference

      SAICSIT '13
      Sponsor:
      • Amazon
      • Rhodes Univ.
      • IBM

      Acceptance Rates

      SAICSIT '13 Paper Acceptance Rate 48 of 89 submissions, 54%;
      Overall Acceptance Rate 187 of 439 submissions, 43%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)An optimized LSTM-based deep learning model for anomaly network intrusion detectionScientific Reports10.1038/s41598-025-85248-z15:1Online publication date: 10-Jan-2025
      • (2024)A Comparative Analysis of Convolutional, Sequential and Their Hybrid Models in Detecting Cyber-Attacks2024 1st International Conference on Trends in Engineering Systems and Technologies (ICTEST)10.1109/ICTEST60614.2024.10576092(01-08)Online publication date: 11-Apr-2024
      • (2024)Unveiling the Landscape of Machine Learning and Deep Learning Methodologies in Network Security: A Comprehensive Literature Review2024 2nd International Conference on Cyber Resilience (ICCR)10.1109/ICCR61006.2024.10533066(1-7)Online publication date: 26-Feb-2024
      • (2023)Fake News Detection Using Feature Extraction, Natural Language Processing, Curriculum Learning, and Deep LearningInternational Journal of Information Technology & Decision Making10.1142/S021962202350034723:03(1063-1098)Online publication date: 6-Apr-2023
      • (2023)IoT Network Intrusion Detection Using Ensemble Learning Approach2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT56998.2023.10307253(1-8)Online publication date: 6-Jul-2023
      • (2023)Arithmetic Optimization With Ensemble Deep Learning SBLSTM-RNN-IGSA Model for Customer Churn PredictionIEEE Access10.1109/ACCESS.2023.330466911(93111-93128)Online publication date: 2023
      • (2023)Predictive Power of XGBoost_BiLSTM Model: A Machine-Learning Approach for Accurate Sleep Apnea Detection Using Electronic Health DataInternational Journal of Computational Intelligence Systems10.1007/s44196-023-00362-y16:1Online publication date: 27-Nov-2023
      • (2023)Machine Learning Techniques in Intrusion Detection System: A SurveyComputer Vision and Robotics10.1007/978-981-19-7892-0_28(365-378)Online publication date: 28-Apr-2023
      • (2022)LSTM-Based Anomalous Behavior Detection in Multi-Agent Reinforcement Learning2022 IEEE International Conference on Cyber Security and Resilience (CSR)10.1109/CSR54599.2022.9850343(16-21)Online publication date: 27-Jul-2022
      • (2022)A Self-adaptive Intrusion Detection Model Based on Bi-LSTM-CRF with Historical Access LogsAdvances in Natural Computation, Fuzzy Systems and Knowledge Discovery10.1007/978-3-030-89698-0_20(185-197)Online publication date: 4-Jan-2022
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media