Abstract
Since the seminal 1998/1999 DARPA evaluations of intrusion detection systems, network attacks have evolved considerably. In particular, after the CodeRed worm of 2001, the volume and sophistication of self-propagating malicious code threats have been increasing at an alarming rate. Many anomaly detectors have been proposed, especially in the past few years, to combat these new and emerging network attacks. At this time, it is important to evaluate existing anomaly detectors to determine and learn from their strengths and shortcomings. In this paper, we evaluate the performance of eight prominent network-based anomaly detectors under malicious portscan attacks. These ADSs are evaluated on four criteria: accuracy (ROC curves), scalability (with respect to varying normal and attack traffic rates, and deployment points), complexity (CPU and memory requirements during training and classification,) and detection delay. These criteria are evaluated using two independently collected datasets with complementary strengths. Our results show that a few of the anomaly detectors provide high accuracy on one of the two datasets, but are unable to scale their accuracy across the datasets. Based on our experiments, we identify promising guidelines to improve the accuracy and scalability of existing and future anomaly detectors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Symantec Internet Security Threat Reports I–XI (January 2002–January 2008)
McAfee Corp., McAfee Virtual Criminology Report: North American Study into Organized Crime and the Internet (2005)
Computer Economics: Economic Impact of Malicious Code Attacks (2001), http://www.computereconomics.com/cei/press/pr92101.html
Williamson, M.M.: Throttling viruses: Restricting propagation to defeat malicious mobile code. In: ACSAC (2002)
Twycross, J., Williamson, M.M.: Implementing and testing a virus throttle. In: Usenix Security (2003)
Sellke, S., Shroff, N.B., Bagchi, S.: Modeling and automated containment of worms. In: DSN (2005)
Jung, J., Paxson, V., Berger, A.W., Balakrishnan, H.: Fast portscan detection using sequential hypothesis testing. In: IEEE Symp. Sec. and Priv. (2004)
Schechter, S.E., Jung, J., Berger, A.W.: Fast detection of scanning worm infections. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 59–81. Springer, Heidelberg (2004)
Weaver, N., Staniford, S., Paxson, V.: Very fast containment of scanning worms. In: Usenix Security (2004)
Chen, S., Tang, Y.: Slowing Down Internet Worms. In: IEEE ICDCS (2004)
Ganger, G., Economou, G., Bielski, S.: Self-Securing Network Interfaces: What, Why, and How. Carnegie Mellon University Technical Report, CMU-CS-02-144 (2002)
Mahoney, M.V., Chan, P.K.: PHAD: Packet Header Anomaly Detection for Indentifying Hostile Network Traffic. Florida Tech. technical report CS-2001-4 (2001)
Mahoney, M.V., Chan, P.K.: Learning Models of Network Traffic for Detecting Novel Attacks. Florida Tech. technical report CS-2002-08 (2002)
Mahoney, M.V., Chan, P.K.: Network Traffic Anomaly Detection Based on Packet Bytes. In: ACM SAC (2003)
Lakhina, A., Crovella, M., Diot, C.: Characterization of network-wide traffic anomalies in traffic flows. In: ACM Internet Measurement Conference (IMC) (2004)
Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: ACM SIGCOMM (2004)
Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. In: ACM SIGCOMM (2005)
Soule, A., Salamatian, K., Taft, N.: Combining Filtering and Statistical methods for anomaly detection. In: ACM/Usenix IMC (2005)
Zou, C.C., Gao, L., Gong, W., Towsley, D.: Monitoring and early warning of Internet worms. In: ACM CCS (2003)
Gu, Y., McCullum, A., Towsley, D.: Detecting anomalies in network traffic using maximum entropy estimation. In: ACM/Usenix IMC (2005)
Next-Generation Intrusion Detection Expert System (NIDES), http://www.csl.sri.com/projects/nides/
Peakflow-SP and Peakflow-X, http://www.arbornetworks.com/peakflowsp , http://www.arbornetworks.com/peakflowx
Cisco IOS Flexible Network Flow, http://www.cisco.com/go/netflow
LBNL/ICSI Enterprise Tracing Project, http://www.icir.org/enterprise-tracing/download.html
WisNet ADS Comparison Homepage, http://wisnet.niit.edu.pk/projects/adeval
Wong, C., Bielski, S., Studer, A., Wang, C.: Empirical Analysis of Rate Limiting Mechanisms. In: Valdes, A., Zamboni, D. (eds.) RAID 2005. LNCS, vol. 3858, pp. 22–42. Springer, Heidelberg (2006)
Shafiq, M.Z., Khayam, S.A., Farooq, M.: Improving Accuracy of Immune-inspired Malware Detectors by using Intelligent Features. In: ACM GECCO (2008)
Ingham, K.L., Inoue, H.: Comparing Anomaly Detection Techniques for HTTP. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 42–62. Springer, Heidelberg (2007)
Lazarevic, A., Ertoz, L., Kumar, V., Ozgur, A., Srivastava, J.: A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection. In: SIAM SDM (2003)
Mueller, P., Shipley, G.: Dragon claws its way to the top. In: Network Computing (2001), http://www.networkcomputing.com/1217/1217f2.html
The NSS Group: Intrusion Detection Systems Group Test (Edition 2) (2001), http://nsslabs.com/group-tests/intrusion-detection-systems-ids-group-test-edition-2.html
Yocom, B., Brown, K.: Intrusion battleground evolves, Network World Fusion (2001), http://www.nwfusion.com/reviews/2001/1008bg.html
Lippmann, R.P., Haines, J.W., Fried, D.J., Korba, J., Das, K.: The 1999 DARPA OffLine Intrusion Detection Evaluation. Comp. Networks 34(2), 579–595 (2000)
Durst, R., Champion, T., Witten, B., Miller, E., Spagnuolo, L.: Testing and Evaluating Computer Intrusion Detection Systems. Comm. of the ACM 42(7), 53–61 (1999)
Shipley, G.: ISS RealSecure Pushes Past Newer IDS Players. In: Network Computing (1999), http://www.networkcomputing.com/1010/1010r1.html
Shipley, G.: Intrusion Detection, Take Two. In: Network Computing (1999), http://www.nwc.com/1023/1023f1.html
Roesch, M.: Snort – Lightweight Intrusion Detection for Networks. In: USENIX LISA (1999)
Lippmann, R.P., Fried, D.J., Graf, I., Haines, J.W., Kendall, K.R., McClung, D., Weber, D., Webster, S.E., Wyschogrod, D., Cunningham, R.K., Zissman, M.A.: Evaluating Intrusion Detection Systems: The 1998 DARPA Off-Line Intrusion Detection Evaluation. In: DISCEX, vol. (2), pp. 12–26 (2000)
DARPA-sponsored IDS Evaluation (1998 and 1999). MIT Lincoln Lab, Cambridge, www.ll.mit.edu/IST/ideval/data/data_index.html
Debar, H., Dacier, M., Wespi, A., Lampart, S.: A workbench for intrusion detection systems. IBM Zurich Research Laboratory (1998)
Denmac Systems, Inc.: Network Based Intrusion Detection: A Review of Technologies (1999)
Ptacek, T.H., Newsham, T.N.: Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection. Secure Networks, Inc. (1998)
Aguirre, S.J., Hill, W.H.: Intrusion Detection Fly-Off: Implications for the United States Navy. MITRE Technical Report MTR 97W096 (1997)
Puketza, N., Chung, M., Olsson, R.A., Mukherjee, B.: A Software Platform for Testing Intrusion Detection Systems. IEEE Software 14(5), 43–51 (1997)
Puketza, N.F., Zhang, K., Chung, M., Mukherjee, B., Olsson, R.A.: A Methodology for Testing Intrusion Detection Systems. IEEE Trans. Soft. Eng. 10(22), 719–729 (1996)
Mell, P., Hu, V., Lippmann, R., Haines, J., Zissman, M.: An Overview of Issues in Testing Intrusion Detection Systems. NIST IR 7007 (2003)
McHugh, J.: The 1998 Lincoln Laboratory IDS Evaluation (A Critique). In: Debar, H., Mé, L., Wu, S.F. (eds.) RAID 2000. LNCS, vol. 1907, Springer, Heidelberg (2000)
Mahoney, M.V., Chan, P.K.: An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection. In: Vigna, G., Krügel, C., Jonsson, E. (eds.) RAID 2003. LNCS, vol. 2820, pp. 220–237. Springer, Heidelberg (2003)
Pang, R., Allman, M., Paxson, V., Lee, J.: The Devil and Packet Trace Anonymization. In: ACM CCR, vol. 36(1) (2006)
Pang, R., Allman, M., Bennett, M., Lee, J., Paxson, V., Tierney, B.: A First Look at Modern Enterprise Traffic. In: ACM/USENIX IMC (2005)
Winpcap homepage, http://www.winpcap.org/
Symantec Security Response, http://securityresponse.symantec.com/avcenter
Shannon, C., Moore, D.: The spread of the Witty worm. IEEE Sec & Priv 2(4), 46–50 (2004)
Axelsson, S.: Intrusion Detection Systems: A Survey and Taxonomy. Technical Report 99-15, Chalmers University (2000)
Ringberg, H., Rexford, J., Soule, A., Diot, C.: Sensitivity of PCA for Traffic Anomaly Detection. In: ACM SIGMETRICS (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ashfaq, A.B., Robert, M.J., Mumtaz, A., Ali, M.Q., Sajjad, A., Khayam, S.A. (2008). A Comparative Evaluation of Anomaly Detectors under Portscan Attacks. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds) Recent Advances in Intrusion Detection. RAID 2008. Lecture Notes in Computer Science, vol 5230. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87403-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-87403-4_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87402-7
Online ISBN: 978-3-540-87403-4
eBook Packages: Computer ScienceComputer Science (R0)