Skip to main content
SpringerLink
Log in

Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Identification of attacks by a network intrusion detection system (NIDS) is an important task. In signature or rule based detection, the previously encountered attacks are modeled, and signatures/rules are extracted. These rules are used to detect such attacks in future, but in anomaly or outlier detection system, the normal network traffic is modeled. Any deviation from the normal model is deemed to be an outlier/ attack. Data mining and machine learning techniques are widely used in offline NIDS. Unsupervised and supervised learning techniques differ the way NIDS dataset is treated. The characteristic features of unsupervised and supervised learning are finding patterns in data, detecting outliers, and determining a learned function for input features, generalizing the data instances respectively. The intuition is that if these two techniques are combined, better performance may be obtained. Hence, in this paper the advantages of unsupervised and supervised techniques are inherited in the proposed hierarchical model and devised into three stages to detect attacks in NIDS dataset. NIDS dataset is clustered using Dirichlet process (DP) clustering based on the underlying data distribution. Iteratively on each cluster, local denser areas are identified using local outlier factor (LOF) which in turn is discretized into four bins of separation based on LOF score. Further, in each bin the normal data instances are modeled using one class classifier (OCC). A combination of Density Estimation method, Reconstruction method, and Boundary methods are used for OCC model. A product rule combination of the threemethods takes into consideration the strengths of each method in building a stronger OCC model. Any deviation from this model is considered as an attack. Experiments are conducted on KDD CUP’99 and SSENet-2011 datasets. The results show that the proposed model is able to identify attacks with higher detection rate and low false alarms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Corona I, Giacinto G, Roli F Adversarial attacks against intrusion detection systems: taxonomy, solutions and open issues. Information Sciences, 2013, 239 (1): 201–225

    Article  Google Scholar 

  2. Sharma A. Cyber wars: a paradigm shift from means to ends. Strategic Analysis, 2010, 34 (1): 62–73

    Article  Google Scholar 

  3. Denning D E. An intrusion-detection model. IEEE Transactions on Software Engineering, 1987, 13 (2): 222–232

    Article  Google Scholar 

  4. Bhuyan M H, Bhattacharyya D K, Kalita J K. Network anomaly detection: methods, systems and tools. IEEE Communications Surveys & Tutorials, 2014, 16(1): 303–336

    Article  Google Scholar 

  5. Davis J J, Clark A J. Data preprocessing for anomaly based network intrusion detection: a review. Computers & Security, 2011, 30(6): 353–375

    Article  Google Scholar 

  6. Wu S Y, Yen E. Data mining-based intrusion detectors. Expert Systems with Applications, 2009, 36(3): 5605–5612

    Article  Google Scholar 

  7. Jiang S Y, Song X, Wang H, Han J J, Li QH. A clustering-based method for unsupervised intrusion detections. Pattern Recognition Letters, 2006, 27(7): 802–810

    Article  Google Scholar 

  8. Helali R GM. Data mining based network intrusion detection system: a survey. In: Sobh T, Elleithy K, Mahmood A, eds. Novel Algorithms and Techniques in Telecommunications and Networking.Springer Netherlands, 2010, 501–505

    Chapter  Google Scholar 

  9. Mukkamala S, Sung A H, Abraham A, Ramos, V. Intrusion detection systems using adaptive regression spines. In: Seruca I, Cordeiro J, Hammoudi S, Filipe J. Enterprise Information Systems VI, 2006, 211–218

    Chapter  Google Scholar 

  10. Tajbakhsh A, Rahmati M, Mirzaei A. Intrusion detection using fuzzy association rules. Applied Soft Computing, 2009, 9(2): 462–469

    Article  Google Scholar 

  11. Su M Y, Yu G J, Lin C Y. A real-time network intrusion detection system for large-scale attacks based on an incremental mining approach. Computers & Security, 2009, 28(5): 301–309

    Article  Google Scholar 

  12. Sangkatsanee P, Wattanapongsakorn N, Charnsripinyo C. Practical real-time intrusion detection using machine learning approaches. Computer Communications, 2011, 34(18): 2227–2235

    Article  Google Scholar 

  13. Sinclair C, Pierce L, Matzner S. An application of machine learning to network intrusion detection. In: Proceedings of the 15th Annual Conference on Computer Security Applications. 1999, 371–377

    Chapter  Google Scholar 

  14. Sommer R, Paxson V. Outside the closed world: on using machine learning for network intrusion detection. In: Proceedings of IEEE Symposium on Security and Privacy. 2010, 305–316

    Google Scholar 

  15. Jain A K. Data clustering: 50 years beyond K-means. Pattern Recognition Letters, 2010, 31(8): 651–666

    Article  Google Scholar 

  16. Mukkamala S, Janoski G, Sung A. Intrusion detection using neural networks and support vector machines. In: Proceedings of the 2002 International Joint Conference on Neural Networks. 2002, 1702–1707

    Google Scholar 

  17. Altwaijry H. Bayesian based intrusion detection system. Lecture Notes in Electrical Engineering, 2012, 170: 29–44

    Article  Google Scholar 

  18. Wuu L C, Hung C H, Chen S F. Building intrusion pattern miner for Snort network intrusion detection system. Journal of Systems and Software, 2007, 80(10): 1699–1715

    Article  Google Scholar 

  19. Sanders C, Smith J. Applied Network Security Monitoring Collection, Detection, and Analysis. Elsevier, 2013

    Google Scholar 

  20. Estevez-Tapiador J M, Garcia-Teodoro P, Diaz-Verdejo J E. Anomaly detection methods in wired networks: a survey and taxonomy. Computer Networks, 2004, 27(16): 1569–1584

    Google Scholar 

  21. Grossman R L. Data Mining: Challenges and Opportunities for Data Mining During the Next Decade, http://www.lac.uic.edu, 1997

    Google Scholar 

  22. Zhang J. Advancements of outlier detection: a survey. ICST Transactions on Scalable Information Systems, 2013, 13(1): 1–26

    Article  Google Scholar 

  23. Freedman D, Pisani R. R. Purves: Statistics. New York: Norton & Co., 1978

    Google Scholar 

  24. Guttõrmsson S E, Marks R J, El-Sharkawi M A, Kerszenbaum I. Elliptical novelty grouping for on-line short-turn detection of excited running rotors. IEEE Transactions on Energy Conversion, 1999, 14(1): 16–22

    Article  Google Scholar 

  25. Aggarwal C C. OnAbnormality Detection in Spuriously Populated Data Streams. In: Proceedings of the 2005 SIAM International Conference on Data Mining. 2005

    Google Scholar 

  26. Abraham B, Box G E P. Bayesian analysis of some outlier problems in time series. Biometrika, 1979, 66(2): 229–236

    Article  MathSciNet  MATH  Google Scholar 

  27. Anderson D, Frivold T, Valdes A. Next Generation Intrusion Detection Expert System (NIDES): A Summary. Menio Park, CA: SRI International, Computer Science Laboratory, 1995

    Google Scholar 

  28. Fawcett T, Provost F. Activity monitoring: noticing interesting changes in behavior. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1999, 53–62

    Google Scholar 

  29. Bishop C. Novelty detection and neural network validation. IEE Proceedings — Vision, Image and Signal Processing, 1994, 141(4): 217–222

    Article  Google Scholar 

  30. Yeung D Y, Chow C. Parzen-window network intrusion detectors. In: Proceedings of the 16th International Conference on Pattern Recognition. 2002, 385–388

    Google Scholar 

  31. Knorr E M, Ng R T. Algorithms for mining distancebased outliers in large datasets. In: Proceedings of the 24th International Conference on Very Large Data Bases. 1998, 392–403

    Google Scholar 

  32. Knorr E M, Ng R T. Finding Intentional Knowledge of Distance-based Outliers. In: Proceedings of the 25th International Conference on Very Large Data Bases. 1999, 211–222

    Google Scholar 

  33. Knorr E M, Ng R T, Tucakov V. Distance-based outliers: algorithms and applications. The VLDB Journal — The International Journal on Very Large Data Bases, 2000, 8(3-4): 237–253

    Article  Google Scholar 

  34. Ramaswamy S, Rastogi R, Shim K. Efficient algorithms for mining outliers from large data sets. ACM SIGMOD Record, 2000, 29(2): 427–438

    Article  Google Scholar 

  35. Breunig MM, Kriegel H P, Ng R T, Sander J. LOF: identifying densitybased local outliers. ACM SIGMOD Record, 2000, 29(2): 93–104

    Article  Google Scholar 

  36. Kriegel H P, Kröger P, Schubert E, Zimek A. LoOP: Local Outlier Probabilities. Proceedings of the 18th ACM conference on Information and knowledge management. 2009, 1649–1652

    Google Scholar 

  37. Papadimitriou S, Kitagawa H, Gibbons P B, Faloutsos C. LOCI: fast outlier detection using the local correlation integral. In: Proceedings of the 19th IEEE International Conference on Data Engineering. 2003, 315–326

    Google Scholar 

  38. Kaufman L, Rousseeuw P J. Finding Groups in Data: An Introduction to Cluster Analysis. New York: John Wiley & Sons, 1990

    Book  Google Scholar 

  39. Ng R T, Han J. Efficient and effective clustering methods for spatial data mining. In: Proceedings of the 20th International Conference on Very Large Data Bases. 1994, pp. 144–155

    Google Scholar 

  40. Guha S, Rastogi R, Shim K. CURE: an efficient clustering algorithm for large databases. In: Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data. 1998, 73–84

    Chapter  Google Scholar 

  41. Khan L, Awad M, Thuraisingham B. A new intrusion detection system using support vector machines and hierarchical clustering. The VLDB Journal — The International Journal on Very Large Data Bases, 2007, 16(4): 507–521

    Article  Google Scholar 

  42. Karypis G, Han E H, Kumar V. CHAMELEON: ahierarchical clustering algorithm using dynamic modeling. Computer, 1999, 32(8): 68–75

    Article  Google Scholar 

  43. Sheikholeslami G, Chatterjee S, Zhang A. WaveCluster: a waveletbased clustering approach for spatial data in very large databases. The VLDB Journal—The International Journal on Very Large Data Bases, 2000, 8(3-4): 289–304

    Article  Google Scholar 

  44. Wang W, Yang J, Muntz R. STING: astatistical information grid approach to spatial data mining. In: Proceedings of the 23rd International Conference on Very Large Data Bases. 1997, 186–195

    Google Scholar 

  45. Zhang J, Hsu W, Lee M L. Clustering in dynamic spatial databases. Journal of Intelligent Information Systems, 2005, 24(1): 5–27

    Article  MATH  Google Scholar 

  46. lachos A, Korhonen A, Ghahramani Z. Unsupervised and constrained Dirichlet process mixture models for verb clustering. In: Proceedings of the Workshop on Geometrical Models of Natural Language Semantics. 2009, 74–82

    Chapter  Google Scholar 

  47. Fan W, Bouguila N, Sallay H. Anomaly intrusion detection using incremental learning of an infinite mixture model with feature selection. Lecture Notes in Computer Science, 2013, 8171: 364–373

    Article  Google Scholar 

  48. Vasudevan, A. R, Selvakumar S. Evolution of a hybrid model using Dirichlet process clustering technique and naive Bayes cassifier for an effective perimeter security device. Technical Report. 2013

    Google Scholar 

  49. Lazarevic A, Ertöz L, Kumar V, Ozgur A, Srivastava J. A comparative study of anomaly detection schemes in network intrusion detection. In: Proceedings of the 2003 SIAM International Conference on Data Mining. 2003, 25–36

    Chapter  Google Scholar 

  50. Zimek A, Campello R J G B, Sander J. Ensembles for unsupervised outlier detection: challenges and research questions a position paper. ACM SIGKDD Explorations Newsletter, 2013, 15(1): 11–22

    Article  Google Scholar 

  51. Garcia S, Luengo J, Sáez J A, López V, Herrera F. A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(4): 734–750

    Article  Google Scholar 

  52. Fayyad U M, Irani K B. Multi-interval discretization of continuousvalued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence. 1993, 1022–1027

    Google Scholar 

  53. Dougherty J, Kohavi R, Sahami M. Supervised and unsupervised discretization of continuous features. In: Proceedings of the 12th International Conference on Machine Learning. 1995, 194–202

    Google Scholar 

  54. Moya M M, Koch M W, Hostetler L D. One-class classifier networks for target recognition applications. In: Proceedings of World Congress on Neural Networks, 1993, 797–801

    Google Scholar 

  55. Tax DMJ, Duin R P W. Combining one-class classifiers. Lecture Notes in Computer Science, 2001, 2096: 299–308

    Article  MathSciNet  MATH  Google Scholar 

  56. Tax D M J. One-class classification, concept learning in the absence of counter examples. Dissertation for the Doctoral Degree. Delft: Delft University of Technology, 2001

    Google Scholar 

  57. Mazhelis O. One-class classifiers: a review and analysis of suitability in the context of mobile-masquerader detection. South African Computer Journal, 2006, 36: 29–48

    Google Scholar 

  58. Hempstalk K, Frank E, Witten I H. One-class classification by combining density and class probability estimation. Lecture Notes in Computer Science, 2008, 5211: 505–519

    Article  Google Scholar 

  59. Giacinto G, Perdisci R, Del Rio M, Roli F. Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Information Fusion, 2008, 9(1): 69–82

    Article  Google Scholar 

  60. Vasudevan A R, Harshini E, Selvakumar S. SSENet-2011: a network intrusion detection system dataset and its comparison with KDD CUP 99 dataset. In: Proceedings of the 2nd IEEE Asian Himalayas International Conference on Internet (AH-ICI 2011). 2011, 1–5

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CDBR-SSE Lab, Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli (NITT), Tiruchirappalli, 620015, India

    Alampallam Ramaswamy Vasudevan & Subramanian Selvakumar

Authors
  1. Alampallam Ramaswamy Vasudevan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subramanian Selvakumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Subramanian Selvakumar.

Additional information

Alampallam Ramaswamy Vasudevan is a PhD research scholar in the Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, Tamil Nadu, India. He received his B. Tech in computer science and engineering from Nehru College of Engineering and Research Center, Thiruvilwamala, India in 2006 and M. Tech in cyber security from Amrita Vishwa Vidyapeetham, Coimbatore, India. His areas of interest include network security, computer networks, and digital forensics.

Subramanian Selvakumar is a professor in the Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, India. He received his PhD from the Indian Institute of Technology Madras (IITM), India in 1999. He has to his credit of publishing 68 research papers. He was the investigator of Rs.1/- Crore research project: Collaborative Directed Basic Research- Smart and Secure Environment (CDBR-SSE) Project sponsored by NTRO, Government of India. His research interests include network security, computer networks, high-speed networks, mobile networks, and wireless sensor networks.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vasudevan, A.R., Selvakumar, S. Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset. Front. Comput. Sci. 10, 755–766 (2016). https://doi.org/10.1007/s11704-015-5116-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-015-5116-8

Keywords

Search

Navigation