Abstract
Collective anomaly is a pattern in the data when a group of similar data instances behave anomalously with respect to the entire dataset. Clustering is a useful unsupervised technique to identify the underlying pattern in the data as well as anomaly detection. However, existing clustering based techniques have high false alarm rates and consider individual data instance behaviour for anomaly detection. In this paper, we formulate the problem of detecting DoS (Denial of Service) attacks as collective anomaly detection and propose a mathematically logical criteria for selecting the important traffic attributes for detecting collective anomaly. Information theoretic co-clustering algorithm is advantageous over regular clustering for creating more fine-grained representation of the data, however lacks the ability to handle mixed attribute data. We extend the co-clustering algorithm by incorporating the ability to handle categorical attributes which augments the detection accuracy of DoS attacks in benchmark KDD cup 1999 network traffic dataset than the existing techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Papalexakis, E.E., Beutel, A., Steenkiste, P.: Network anomaly detection using co-clustering. In: Proceedings of the 2012 International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2012), pp. 403–410. IEEE Computer Society, Washington, DC (2012)
Symantec Internet Security Threat Report. http://www.symantec.com/content/en/us/enterprise/
Stuxnet. http://www.stuxnet.net/
Verizon’s Data Breach Investigation Report 2014. http://www.verizonenterprise.com/DBIR/2014/
Google hack attack was ultra sophisticated, new details show. http://www.wired.com/2010/01/operation-aurora/
Ahmed, M., Naser, A.: A novel approach for outlier detection and clustering improvement. In: 2013 8th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 577–582 (2013)
Goldstein, M., Mennatallah A.: Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. In: Proceedings of the 3rd RapidMiner Community Meeting and Conference (RCOMM 2012), pp. 1–12. Shaker Verlag GmbH, Aachen, August 2012
He, Z., Xu, X., Deng, S.: Discovering cluster based local outliers. Pattern Recogn. Lett. 2003, 9–10 (2003)
Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2003, pp. 89–98. ACM, New York (2003)
Banerjee, A., Dhillon, I., Ghosh, J., Merugu, S., Modha, D.S.: A generalized maximum entropy approach to Bregman co-clustering and matrix approximation. J. Mach. Learn. Res. 8, 1919–1986 (2007)
1999 kdd cup dataset. www.kdd.ics.uci.edu
Ahmed, M., Mahmood, A., Hu, J.: Outlier detection. In: The State of the Art in Intrusion Prevention and Detection, pp. 3–23. CRC Press, USA
Ahmed, M., Mahmood, A.: Network traffic analysis based on collective anomaly detection. In: 2014 IEEE 9th Conference on Industrial Electronics and Applications (ICIEA), pp. 1141–1146, June 2014
Kendall, K.: A database of computer attacks for the evaluation of intrusion detection systems. In: DARPA Off-line Intrusion Detection Evaluation, Proceedings of DARPA Information Survivality Conference and Eexposition (DISCEX), pp. 12–26 (1999)
How to survive botnet attacks - Understanding Botnets and DDOS attacks. https://www.youtube.com
Portnoy, L., Eskin, E., Stolfo, S.: Intrusion detection with unlabeled data using clustering. In: Proceedings of ACM CSS Workshop on Data Mining Applied to Security (DMSA-2001), pp. 5–8 (2001)
Valdes, A., Javitz, H.S.: The nides statistical component: description and justification. Technical report (1993)
Ahmed, M., Mahmood, A.N., Islam, M.R.: A survey of anomaly detection techniques in financial domain. Future Generation Computer Systems (2015)
Levin, I.: KDD-99 classifier learning contest LLSoft’s results overview. SIGKDD Explor. Newsl. 1(2), 67–75 (2000)
Agarwal, R., Joshi, M.V.: Pnrule: A new framework for learning classifier models in data mining (a case-study in network intrusion detection). Technical report, IBM Research Report, Computer Science/Mathematics (2000)
Boriah, S., Chandola, V., Kumar, V.: Similarity measures for categorical data: a comparative evaluation. In: Proceedings of the Eighth SIAM International Conference on Data Mining, pp. 243–254
Leung, K., Leckie, C.: Unsupervised anomaly detection in network intrusion detection using clusters. In: Proceedings of the Twenty-Eighth Australasian Conference on Computer Science, ACSC 2005, vol. 38, pp. 333–342. Australian Computer Society Inc, Darlinghurst (2005)
Syarif, I., Prugel-Bennett, A., Wills, G.: Unsupervised clustering approach for network anomaly detection. In: Benlamri, R. (ed.) NDT 2012, Part I. CCIS, vol. 293, pp. 135–145. Springer, Heidelberg (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ahmed, M., Mahmood, A.N. (2015). Network Traffic Pattern Analysis Using Improved Information Theoretic Co-clustering Based Collective Anomaly Detection. In: Tian, J., Jing, J., Srivatsa, M. (eds) International Conference on Security and Privacy in Communication Networks. SecureComm 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 153. Springer, Cham. https://doi.org/10.1007/978-3-319-23802-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-23802-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23801-2
Online ISBN: 978-3-319-23802-9
eBook Packages: Computer ScienceComputer Science (R0)