Skip to main content
SpringerLink
Log in
Book cover

International Conference on Information Security Practice and Experience

ISPEC 2013: Information Security Practice and Experience pp 1–15Cite as

Enhancing False Alarm Reduction Using Pool-Based Active Learning in Network Intrusion Detection

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 7863))

Abstract

Network intrusion detection systems (NIDSs) are an important and essential defense mechanism against network attacks. However, during their detection, a large number of NIDS false alarms could be generated, which is a major challenging problem for these systems. To mitigate this issue, machine-learning based false alarm filters have been developed to refine false alarms, but it is very laborious and difficult for security experts to provide many labeled examples to train a classifier. In this paper, we therefore attempt to investigate the performance of active learning, which can make the optimal use of the given datasets, in this particular field of NIDS false alarm reduction. After analyzing the relationship between the process of false alarm reduction and the process of intrusion detection, we design a simple but efficient pool-based active learning algorithm in a false alarm filter and evaluate its performance by comparing it with several traditional supervised machine learning algorithms. The experimental results show that the designed pool-based active learner can generally achieve a better outcome than a traditional machine learning algorithm, and that the designed scheme can approximatively reduce the required number of labeled alarms by half.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alharby, A., Imai, H.: IDS False Alarm Reduction Using Continuous and Discontinuous Patterns. In: Ioannidis, J., Keromytis, A.D., Yung, M. (eds.) ACNS 2005. LNCS, vol. 3531, pp. 192–205. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  2. Almgren, M., Jonsson, E.: Using Active Learning in Intrusion Detection. In: Proceedings of the 17th IEEE Computer Security Foundations Workshop (CSFW), pp. 88–98 (2004)

    Google Scholar 

  3. Axelsson, S.: The Base-rate Fallacy and the Difficulty of Intrusion Detection. ACM Transactions on Information and System Security, 186–205 (August 2000)

    Google Scholar 

  4. DARPA: KDD Cup 1999 Data, http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html

  5. Ghosh, A.K., Wanken, J., Charron, F.: Detecting Anomalous and Unknown Intrusions Against Programs. In: Proceedings of the 1998 Annual Computer Security Applications Conference (ACSAC), pp. 259–267 (1998)

    Google Scholar 

  6. Görnitz, N., Kloft, M., Rieck, K., Brefeld, U.: Active Learning for Network Intrusion Detection. In: Proceedings of the 2nd ACM Workshop on Security and Artificial Intelligence (AISec), pp. 47–54 (2009)

    Google Scholar 

  7. Law, K.H., Kwok, L.F.: IDS False Alarm Filtering Using KNN Classifier. In: Lim, C.H., Yung, M. (eds.) WISA 2004. LNCS, vol. 3325, pp. 114–121. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  8. Lee, W., Stolfo, S.J., Mok, K.W.: A Data Mining Framework for Building Intrusion Detection Models. In: Proc. of the 1999 IEEE Symposium on Security and Privacy, pp. 120–132 (1999)

    Google Scholar 

  9. Li, Y., Guo, L.: An Active Learning based TCM-KNN Algorithm for Supervised Network Intrusion Detection. Computers and Security 26(7-8), 459–467 (2007)

    Article  Google Scholar 

  10. Lippmann, R.P., et al.: Evaluating Intrusion Detection Systems: the 1998 DARPA off-line Intrusion Detection Evaluation. In: Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX), pp. 12–26 (2000)

    Google Scholar 

  11. McCallum, A., Nigam, K.: Employing EM and Pool-Based Active Learning for Text Classification. In: Proceedings of the 15th International Conference on Machine Learning (ICML), pp. 350–358 (1998)

    Google Scholar 

  12. McHugh, J.: Testing Intrusion Detection Systems: a Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory. ACM Transactions on Information System Security, 262–294 (2000)

    Google Scholar 

  13. Meng, Y., Kwok, L.-f.: Adaptive False Alarm Filter Using Machine Learning in Intrusion Detection. In: Wang, Y., Li, T. (eds.) Practical Applications of Intelligent Systems. AISC, vol. 124, pp. 573–584. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  14. Meng, Y., Li, W.: Constructing Context-based Non-Critical Alarm Filter in Intrusion Detection. In: Proceedings of the 7th International Conference on Internet Monitoring and Protection (ICIMP), pp. 75–81 (2012)

    Google Scholar 

  15. Meng, Y., Li, W., Kwok, L.-f.: Intelligent Alarm Filter Using Knowledge-based Alert Verification in Network Intrusion Detection. In: Chen, L., Felfernig, A., Liu, J., Raś, Z.W. (eds.) ISMIS 2012. LNCS, vol. 7661, pp. 115–124. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  16. Pietraszek, T.: Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 102–124. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  17. Roesch, M.: Snort: Lightweight Intrusion Detection for Networks. In: Proceedings of the 13th Large Installation System Administration Conference (LISA), pp. 229–238 (1999)

    Google Scholar 

  18. Scarfone, K., Mell, P.: Guide to Intrusion Detection and Prevention Systems (IDPS), pp. 800–894. NIST Special Publication (2007), http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

  19. Seliya, N., Khoshgoftaar, T.M.: Active Learning with Neural Networks for Intrusion Detection. In: Proceedings of the 2010 IEEE International Conference on Information Reuse and Integration (IRI), pp. 49–54 (2010)

    Google Scholar 

  20. Snort. (May 2012), http://www.snort.org/

  21. Sommer, R., Paxson, V.: Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In: Proceedings of the 2010 IEEE Symposium on Security and Privacy, pp. 305–316 (2010)

    Google Scholar 

  22. Stokes, J.W., Platt, J.C.: ALADIN: Active Learning of Anomalies to Detect Intrusion. Technique Report. Microsoft Network Security Redmond, WA 98052 USA (2008)

    Google Scholar 

  23. Symantec Corp., Internet Security Threat Report, vol. 16 (July 2012), http://www.symantec.com/business/threatreport/index.jsp

  24. Valdes, A., Anderson, D.: Statistical Methods for Computer Usage Anomaly Detection Using NIDES. Technical Report, SRI International (January 1995)

    Google Scholar 

  25. Vigna, G., Kemmerer, R.A.: NetSTAT: a Network-based Intrusion Detection Approach. In: Proceedings of the 1998 Annual Computer Security Applications Conference (ACSAC), pp. 25–34. IEEE Press, New York (1998)

    Google Scholar 

  26. Wireshark, (May 2012), http://www.wireshark.org

  27. Zhou, Z.-H., Chen, K.-J., Dai, H.-B.: Enhancing Relevance Feedback in Image Retrieval using Unlabeled Data. ACM Transactions on Information Systems 24(2), 219–244 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, City University of Hong Kong, Hong Kong, China

    Yuxin Meng & Lam-For Kwok

Authors
  1. Yuxin Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lam-For Kwok
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Information Systems, Singapore Management University, 178902, Singapore, Singapore

    Robert H. Deng

  2. School of Computer and Communication, University of technology, 730050, Lanzhou, China

    Tao Feng

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Meng, Y., Kwok, LF. (2013). Enhancing False Alarm Reduction Using Pool-Based Active Learning in Network Intrusion Detection. In: Deng, R.H., Feng, T. (eds) Information Security Practice and Experience. ISPEC 2013. Lecture Notes in Computer Science, vol 7863. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38033-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38033-4_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38032-7

  • Online ISBN: 978-3-642-38033-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation