skip to main content
10.1145/2345396.2345484acmotherconferencesArticle/Chapter ViewAbstractPublication PagesacciciConference Proceedingsconference-collections
research-article

An effective unsupervised network anomaly detection method

Published:03 August 2012Publication History

ABSTRACT

In this paper, we present an effective tree based subspace clustering technique (TreeCLUS) for finding clusters in network intrusion data and for detecting unknown attacks without using any labelled traffic or signatures or training. To establish its effectiveness in finding all possible clusters, we perform a cluster stability analysis. We also introduce an effective cluster labelling technique (CLUSLab) to generate labelled dataset based on the stable cluster set generated by TreeCLUS. CLUSLab is a multi-objective technique that exploits an ensemble approach for stability analysis of the clusters generated by TreeCLUS. We evaluate the performance of both TreeCLUS and CLUSLab in terms of several real world intrusion datasets to identify unknown attacks and find that both outperform the competing algorithms.

References

  1. A. Patcha and J. M. Park. An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks, 51(12):3448--3470, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. M.-Y. Su. Using clustering to improve the KNN-based classifiers for online anomaly network traffic identification. J. Netw. Comput. Appl., 34(2):722--730, March 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. N. Toosi and M. Kahani. A new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers. Comput. Commun., 30(10):2201--2212, July 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Comput. Surv., 41(3):15:1--15:58, July 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. SNORT. open source network intrusion prevention and detection system. http://www.snort.org/.Google ScholarGoogle Scholar
  6. BRO. Unix-based network intrusion detection system. http://bro-ids.org/.Google ScholarGoogle Scholar
  7. M. Tavallaee, N. Stakhanova, and A. Ghorbani. Toward credible evaluation of anomaly-based intrusion-detection methods. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 40(5):516--524, sept. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. L. Portnoy, E. Eskin, and S. Stolfo. Intrusion detection with unlabeled data using clustering, 2001.Google ScholarGoogle Scholar
  9. K. Leung and C. Leckie. Unsupervised anomaly detection in network intrusion detection using clusters. In Proceedings of the Twenty-eighth Australasian conference on Computer Science - Volume 38, pages 333--342, Darlinghurst, Australia, Australia, 2005. Australian Computer Society, Inc. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita. NADO: network anomaly detection using outlier approach. In Proceedings of the International Conference on Communication, Computing & Security, pages 531--536, New York, NY, USA, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Jiang, X. Song, H. Wang, J.-J. Han, and Q.-H. Li. A clustering-based method for unsupervised intrusion detections. Pattern Recogn. Lett., 27(7):802--810, May 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. J. Song, H. Takakura, Y. Okabe, and K. Nakao. Toward a more practical unsupervised anomaly detection system. Information Sciences, Aug 2011.Google ScholarGoogle Scholar
  13. P. Casas, J. Mazel, and P. Owezarski. Unsupervised network intrusion detection systems: Detecting the unknown without knowledge. Computer Communications, Jan 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. T. Lange, V. Roth, M. L. Braun, and J. M. Buhmann. Stability-based validation of clustering solutions. Neural Comput., 16(6):1299--1323, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. Ben-David, U. v. Luxburg, and D. Pál. A sober look at clustering stability. In COLT, pages 5--19, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. K. Das and J. Sil. Cluster validation method for stable cluster formation. Canadian Journal on Artificial Intelligence, Machine Learning and Pattern Recognition, 1(3):26--41, July 2010.Google ScholarGoogle Scholar
  17. M. Halkidi, Y. Batistakis, and M. Vazirgiannis. On clustering validation techniques. J. Intell. Inf. Syst., 17(2--3):107--145, December 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. Jun. An ensemble method for validation of cluster analysis. International Journal of Computer Science Issues, 8(6):26--30, September 2011.Google ScholarGoogle Scholar
  19. F. Amiri, M. M. R. Yousefi, C. Lucas, A. Shakery, and N. Yazdani. Mutual information-based feature selection for intrusion detection systems. Journal of Network and Computer Applications, 34(4):1184--1199, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. J. Dunn. Well separated clusters and optimal fuzzy partitions. Journal of Cybernetics, 4:95--104, 1974.Google ScholarGoogle ScholarCross RefCross Ref
  21. L. Hubert and J. Schultz. Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29(2):190--241, 1976.Google ScholarGoogle ScholarCross RefCross Ref
  22. D. L. Davies and D. W. Bouldin. A cluster separation measure. IEEE Transaction on Pattern Analysis and Machine Intelligence, 1(2):224--227, 1979. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20(1):53--65, 1987. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. X. L. Xie and G. Beni. A validity measure for fuzzy clustering. IEEE Transactions on Pattern Analysis and machine Intelligence, 13(4):841--847, 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Frank and A. Asuncion. UCI Machine Learning Repository {http://archive.ics.uci.edu/ml}, 2010. University of California, School of Information and Computer Sciences, Irvine, CA.Google ScholarGoogle Scholar
  26. TUIDS12. TUIDS network intrusion datasets. http://tezu.ernet.in/~dkb/resource.html, May04, 2012.Google ScholarGoogle Scholar
  27. KDDCUP99. Winning strategy in KDD99. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html, October 28 1999.Google ScholarGoogle Scholar
  28. R. Beghdad. Critical study of supervised learning techniques in predicting attacks. Information Security Journal: A Global Perspective, 19(1):22--35, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. C.-F. Tsai and C.-Y. Lin. A triangle area based nearest neighbors approach to intrusion detection. Pattern Recogn., 43(1):222--229, January 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. S. J. Horng, M. Y. Su, Y. H. Chen, T. W. Kao, R. J. Chen, J. L. Lai, and C. D. Perkasa. A novel intrusion detection system based on hierarchical clustering and support vector machines. Expert Syst. Appl., 38(1):306--313, January 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An effective unsupervised network anomaly detection method

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in
          • Published in

            cover image ACM Other conferences
            ICACCI '12: Proceedings of the International Conference on Advances in Computing, Communications and Informatics
            August 2012
            1307 pages
            ISBN:9781450311960
            DOI:10.1145/2345396

            Copyright © 2012 ACM

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 3 August 2012

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader