Abstract
Intrusion detection systems are one of the security tools widely deployed in network architectures in order to monitor, detect and eventually respond to any suspicious activity in the network. However, the constantly growing complexity of networks and the virulence of new attacks require more adaptive approaches for optimal responses. In this work, we propose a semi-supervised approach for network anomaly detection inspired from the biological negative selection process. Based on a reduced dataset with a filter/ranking feature selection technique, our algorithm, namely negative selection for network anomaly detection (NSNAD), generates a set of detectors and uses them to classify events as anomaly. Otherwise, they are matched against an Artificial Human Leukocyte Antigen in order to be classified as normal. The accuracy and the computational time of NSNAD are tested under three intrusion detection datasets: NSL-KDD, Kyoto2006+ and UNSW-NB15. We compare the performance of NSNAD against a fully supervised algorithm (Naïve Bayes), an unsupervised clustering algorithm (K-means) and a semi-supervised algorithm (One-class SVM) with respect to multiple accuracy metrics. We also compare the time incurred by each algorithm in training and classification stages.
Similar content being viewed by others
Notes
Any disease-producing agent, especially a virus, bacterium, or other microorganism.
In k-fold cross-validation, the original sample is randomly partitioned into k equal-sized subsamples. Of the k subsamples, a single one is retained as test data, and the remaining \(k - 1\) subsamples are used as training data. The cross-validation process is then repeated k times, with each of the k subsamples used exactly once as test data. The k results from the folds are then averaged to produce a single estimation.
References
Abas EAER, Abdelkader H, Keshk A (2015) Artificial immune system based intrusion detection. In: 2015 IEEE seventh international conference on intelligent computing and information systems (ICICIS), pp 542–546. Institute of Electrical & Electronics Engineers (IEEE). https://doi.org/10.1109/intelcis.2015.7397274
Agrawal A, Mohammed S, Fiaidhi J (2016) Developing data mining techniques for intruder detection in network traffic. Int J Secur Appl 10(8):335–342. https://doi.org/10.14257/ijsia.2016.10.8.29
Al-Enezi J, Abbod M, Alsharhan S (2010) Artificial immune systems-models, algorithms and applications. http://www.arpapress.com/Volumes/Vol3Issue2/IJRRAS_3_2_01.pdf
Ambusaidi MA, He X, Nanda P, Tan Z (2016) Building an intrusion detection system using a filter-based feature selection algorithm. IEEE Trans Comput 65(10):2986–2998. https://doi.org/10.1109/TC.2016.2519914
Amer SH, Hamilton J (2010) Intrusion detection systems (ids) taxonomy-a short review. Def Cyber Secur 13(2):23–30
Ammar A (2015) Comparison of feature reduction techniques for the binominal classification of network traffic. J Data Anal Inf Process 3(02):11. https://doi.org/10.4236/jdaip.2015.32002
Anusha K, Sathiyamoorthy E (2016) Omamids: ontology based multi-agent model intrusion detection system for detecting web service attacks. J Appl Secur Res 11(4):489–508. https://doi.org/10.1080/19361610.2016.1211847
Axelsson S (2000) Intrusion detection systems: a survey and taxonomy. Report, Technical report
Bahl S, Sharma SK (2016) A minimal subset of features using correlation feature selection model for intrusion detection system. In: Proceedings of the second international conference on computer and communication technologies, pp 337–346. Springer. https://doi.org/10.1007/978-81-322-2523-2_32
Bethi SK, Phoha VV, Reddy YM (2004) Clique clustering approach to detect denial-of-service attacks. In: Proceedings from the fifth annual IEEE SMC information assurance workshop 2004, pp 447–448. https://doi.org/10.1109/iaw.2004.1437856
Bhuyan M, Bhattacharyya D, Kalita J (2014) Network anomaly detection: methods, systems and tools. Commun Surv Tutor IEEE 16(1):1–34
Brownlee J (2011) Clever algorithms: nature-inspired programming recipes. Jason Brownlee
Buitinck L, Louppe G, Blondel M, Pedregosa F, Mueller A, Grisel O, Niculae V, Prettenhofer P, Gramfort A, Grobler J, Layton R, VanderPlas J, Joly A, Holt B, Varoquaux G (2013) API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD workshop: languages for data mining and machine learning, pp 108–122
Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167. https://doi.org/10.1023/a:1009715923555
de Castro L, Zuben FV (2002) Learning and optimization using the clonal selection principle. IEEE Trans Evol Comput 6(3):239–251. https://doi.org/10.1109/tevc.2002.1011539
de Castro LN, Timmis JI (2003) Artificial immune systems as a novel soft computing paradigm. Soft Comput 7(8):526–544
Cemerlic A, Yang L, Kizza JM (2008) Network intrusion detection based on bayesian networks. In: SEKE, pp 791–794
Chan FT, Prakash A, Tibrewal R, Tiwari M (2013) Clonal selection approach for network intrusion detection. In: Proceedings of the 3rd international conference on intelligent computational systems (ICICS’2013), Singapore, pp 1–5
Chen MH, Chang PC, Wu JL (2016) A population-based incremental learning approach with artificial immune system for network intrusion detection. Eng Appl Artif Intell 51:171–181. https://doi.org/10.1016/j.engappai.2016.01.020
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1007/bf00994018
Crosbie M, Spafford G (1995) Applying genetic programming to intrusion detection. In: Working notes for the AAAI symposium on genetic programming, pp 1–8. MIT Press, Cambridge
DasGupta D (1993) An overview of artificial immune systems and their applications. In: Artificial immune systems and their applications, pp 3–21. Springer
Dasgupta D, Nino F (2008) Immunological computation: theory and applications. CRC Press, Boca Raton
Dasgupta D, Yu S, Nino F (2011) Recent advances in artificial immune systems: models and applications. Appl Soft Comput 11(2):1574–1587. https://doi.org/10.1016/j.asoc.2010.08.024
Dhanabal L, Shantharajah S (2015) A study on NSL-KDD dataset for intrusion detection system based on classification algorithms. Int J Adv Res Comput Commun Eng 4(6):446–452
Ding K, Li J, Liu H (2019) Interactive anomaly detection on attributed networks. In: In the twelfth ACM international conference on web search and data mining (WSDM ’19). https://doi.org/10.1145/3289600.3290964
Empirical rule: What is it? (2017). http://www.statisticshowto.com/empirical-rule-2/
Forrest S, Perelson A, Allen L, Cherukuri R (1994) Self-nonself discrimination in a computer. In: Proceedings of 1994 IEEE computer society symposium on research in security and privacy, p 202. Institute of Electrical & Electronics Engineers (IEEE). https://doi.org/10.1109/risp.1994.296580
Gentile C, Li S, Kar P, Karatzoglou A, Zappella G, Etrue E (2017) On context-dependent clustering of bandits. In: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning, proceedings of machine learning research, vol 70, pp 1253–1262. PMLR, International Convention Centre, Sydney, Australia. http://proceedings.mlr.press/v70/gentile17a.html
Ghanem TF, Elkilani WS, Abdul-kader HM (2015) A hybrid approach for efficient anomaly detection using metaheuristic methods. J Adv Res 6(4):609–619. https://doi.org/10.1016/j.jare.2014.02.009
González-Pino J, Edmonds J, Papa M (2006) Attribute selection using information gain for a fuzzy logic intrusion detection system. In: Defense and security symposium, pp 62410D–62410D. International society for optics and photonics
González FA, Dasgupta D (2003) Anomaly detection using real-valued negative selection. Genet Program Evolvable Mach 4(4):383–403
Guha S, Yau SS, Buduru AB (2016) Attack detection in cloud infrastructures using artificial neural network with genetic feature selection. In: Dependable, autonomic and secure computing, 14th International conference on pervasive intelligence and computing, 2nd International conf on big data intelligence and computing and cyber science and technology congress (DASC/PiCom/DataCom/CyberSciTech), 2016 IEEE 14th Intl C, pp 414–419. IEEE
Guo H, Feng Y, Hao F, Zhong S, Li S (2014) Dynamic fuzzy logic control of genetic algorithm probabilities. J Comput 9(1):22–27. https://doi.org/10.4304/jcp.9.1.22-27
Gutierrez MP, Kiekintveld C (2016) Bandits for cybersecurity: adaptive intrusion detection using honeypots. In: AAAI Workshop: Artificial Intelligence for Cyber Security
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software. SIGKDD Explor Newsl 11(1):10. https://doi.org/10.1145/1656274.1656278
Hao F, Li S, Min G, Kim HC, Yau SS, Yang LT (2015) An efficient approach to generating location-sensitive recommendations in ad-hoc social network environments. IEEE Trans Serv Comput 8(3):520–533. https://doi.org/10.1109/tsc.2015.2401833
Hao F, Park DS, Li S, Lee HM (2016) Mining \(\lambda\)-maximal cliques from a fuzzy graph. Sustainability 8(6):553
Hofmann A, Horeis T, Sick B (2004) Feature selection for intrusion detection: an evolutionary wrapper approach. In: 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), vol 2, pp 1563–1568. Institute of Electrical & Electronics Engineers (IEEE). https://doi.org/10.1109/ijcnn.2004.1380189
Hofmeyr SA, Forrest S (2000) Architecture for an artificial immune system. Evol Comput 8(4):443–473. https://doi.org/10.1162/106365600568257
Hong L (2008) Artificial immune system for anomaly detection. In: 2008 IEEE international symposium on knowledge acquisition and modeling workshop, pp 340–343. Institute of Electrical & Electronics Engineers (IEEE). https://doi.org/10.1109/kamw.2008.4810493
Hoque MS, Mukit M, Bikas M, Naser A, et al. (2012) An implementation of intrusion detection system using genetic algorithm. arXiv preprint arXiv:1204.1336
Igbe O, Darwish I, Saadawi T (2016) Distributed network intrusion detection systems: an artificial immune system approach. In: Connected health: applications, systems and engineering technologies (CHASE), 2016 IEEE First International Conference on, pp 101–106. IEEE
Janarthanan T, Zargari S (2017) Feature selection in unsw-nb15 and kddcup’99 datasets. In: 2017 IEEE 26th international symposium on industrial electronics (ISIE), pp 1881–1886. IEEE
Kar P, Li S, Narasimhan H, Chawla S, Sebastiani F (2016) Online optimization methods for the quantification problem. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1625–1634. ACM
Karegowda AG, Manjunath A, Jayaram M (2010) Comparative study of attribute selection using gain ratio and correlation based feature selection. Int J Inf Technol Knowl Manag 2(2):271–277
Kayacik HG, Zincir-Heywood AN, Heywood MI (2005) Selecting features for intrusion detection: A feature relevance analysis on kdd 99 intrusion detection datasets. In: Proceedings of the third annual conference on privacy, security and trust
Khammassi C, Krichen S (2017) A GA-LR wrapper approach for feature selection in network intrusion detection. Comput Secur 70:255–277
Kim J, Bentley PJ (2001) Towards an artificial immune system for network intrusion detection: An investigation of clonal selection with a negative selection operator. In: Proceedings of the 2001 congress on evolutionary computation, 2001. vol 2, pp 1244–1252. IEEE
Kim J, Bentley PJ (2002) Towards an artificial immune system for network intrusion detection: an investigation of dynamic clonal selection. In: Proceedings of the 2002 congress on evolutionary computation, 2002. CEC’02., vol 2, pp 1015–1020. IEEE
Kira K, Rendell LA (1992) A practical approach to feature selection. In: Proceedings of the ninth international workshop on Machine learning, pp 249–256
Korda N, Szörényi B, Shuai L (2016) Distributed clustering of linear bandits in peer to peer networks. In: Journal of machine learning research workshop and conference proceedings, vol 48, pp 1301–1309. International Machine Learning Society
Kumar V, Chauhan H, Panwar D (2013) K-means clustering approach to analyze NSL-KDD intrusion detection dataset. International Journal of Soft Computing and Engineering (IJSCE) ISSN, pp 2231–2307
Li S, Hao F, Li M, Kim HC (2013) Medicine rating prediction and recommendation in mobile social networks. In: International conference on grid and pervasive computing, pp 216–223. Springer
Li S, Karatzoglou A, Gentile C: Collaborative filtering bandits. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval
Li X, Ye N (2001) Decision tree classifiers for computer intrusion detection. J Parallel Distrib Comput Pract 4(2):179–190
Lu C, Feng J, Lin Z, Mei T, Yan S (2018) Subspace clustering by block diagonal representation. IEEE Transactions on Pattern Analysis and Machine Intelligence pp 1–1. https://doi.org/10.1109/tpami.2018.2794348
Lu W, Traore I (2004) Detecting new forms of network intrusion using genetic programming. Comput Intell 20(3):475–494
Matthews BW (1975) Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405(2):442–451
Mohammadi M, Akbari A, Raahemi B, Nassersharif B, Asgharian H (2014) A fast anomaly detection system using probabilistic artificial immune algorithm capable of learning new attacks. Evol Intel 6(3):135–156. https://doi.org/10.1007/s12065-013-0101-3
Moustafa JSN (2016) The unsw-nb15 data set description. https://www.unsw.adfa.edu.au/australian-centre-for-cyber-security/cybersecurity/ADFA-NB15-Datasets/
Moustafa N, Slay J (2015) The significant features of the unsw-nb15 and the kdd99 data sets for network intrusion detection systems. Unpublished. https://doi.org/10.13140/RG.2.1.2264.4883
Moustafa N, Slay J (2015) UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military communications and information systems conference (MilCIS), pp 1–6. IEEE. https://doi.org/10.1109/milcis.2015.7348942
Moustafa N, Slay J (2016) The evaluation of network anomaly detection systems: statistical analysis of the unsw-nb15 data set and the comparison with the kdd99 data set. Inf Secur J Global Perspect 25:1–3. https://doi.org/10.1080/19393555.2015.1125974
Mukkamala S, Janoski G, Sung A (2002) Intrusion detection using neural networks and support vector machines. In: Neural Networks, 2002. IJCNN’02. In: Proceedings of the 2002 international joint conference on, vol 2, pp 1702–1707. IEEE
Najafabadi MM, Khoshgoftaar TM, Seliya N (2016) Evaluating feature selection methods for network intrusion detection with kyoto data. Int J Reliab Qual Saf Eng 23(01):1650001. https://doi.org/10.1142/s0218539316500017
Nastaiinullah, N., Adiwijaya, Kurniati, AP (2014) Anomaly detection on intrusion detection system using CLIQUE partitioning. In: 2014 2nd International conference on information and communication technology (ICoICT). IEEE. https://doi.org/10.1109/icoict.2014.6914031
Nguyen HT, Petrović S, Franke K (2010) A comparison of feature-selection methods for intrusion detection, pp 242–255. Springer. https://doi.org/10.1007/978-3-642-14706-7_19
Noble CC, Cook DJ (2003) Graph-based anomaly detection. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM Press. https://doi.org/10.1145/956750.956831
Owen JA, Punt J, Stranford SA et al (2013) Kuby immunology. WH Freeman, New York
Panda M, Patra MR (2007) Network intrusion detection using naive bayes. Int J Comput Sci Netw Secur 7(12):258–263
Parham P (2015) The immune system, 4th edn. Garland Science, New York City
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2007–2017) Scikit-learn tool. http://scikit-learn.org
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
Popoola E, Adewumi AO (2017) Efficient feature selection technique for network intrusion detection system using discrete differential evolution and decision. IJ Netw Secur 19(5):660–669
Portnoy L (2000) Intrusion detection with unlabeled data using clustering
Rathore H (2016) Mapping biological systems to network systems
Ryan J, Lin MJ, Miikkulainen R (1998) Intrusion detection with neural networks. In: Proceedings of the advances in neural information processing systems 10: annual conference on neural information processing systems 1997, NeurIPS 1977, Denver, Colorado, USA, 1997. The MIT Press 1998, ISBN 0-262-10076-2
Salamatova T, Zhukov V (2017) Network intrusion detection by the coevolutionary immune algorithm of artificial immune systems with clonal selection. IOP Conf Ser Mater Sci Eng 173(1):012016
Saurabh P, Verma B (2016) An efficient proactive artificial immune system based anomaly detection and prevention system. Expert Syst Appl 60:311–320
Seresht NA, Azmi R (2014) MAIS-IDS: a distributed intrusion detection system using multi-agent ais approach. Eng Appl Artif Intell 35:286–298
Shanmugavadivu R, Nagarajan N (2011) Network intrusion detection system using fuzzy logic. Indian J Comput Sci Eng (IJCSE) 2(1):101–111
Shen J, Wang J, Ai H (2012) An improved artificial immune system-based network intrusion detection by using rough set. CN 04(01):41–47. https://doi.org/10.4236/cn.2012.41006
Shon T, Moon J (2007) A hybrid machine learning approach to network anomaly detection. Inf Sci 177(18):3799–3821
Sompayrac LM (2016) How the immune system works. The how it works series, 5ed edn. Wiley, Hoboken
Song J, Takakura H, Okabe Y, Eto M, Inoue D, Nakao K (2011) Statistical analysis of honeypot data and building of kyoto 2006+ dataset for NIDS evaluation. In: Proceedings of the first workshop on building analysis datasets and gathering experience returns for security, pp 29–36. ACM. https://doi.org/10.1145/1978672.1978676
Souici-Meslati L, Zekri M (2016) Immunological approach for intrusion detection. REVUE AFRICAINE DE LA RECHERCHE EN INFORMATIQUE ET MATHÉMATIQUES APPLIQUÉES 17:
Sridevi R, Chattemvelli R (2012) Genetic algorithm and artificial immune systems: a combinational approach for network intrusion detection. In: 2012 International Conference on Advances in Engineering, Science and Management (ICAESM), pp 494–498. IEEE
Tabatabaefar M, Miriestahbanati M, Grégoire JC (2017) Network intrusion detection through artificial immune system. In: Systems Conference (SysCon), 2017 Annual IEEE International, pp 1–6. IEEE
Tavallaee M, Bagheri E, Lu W, Ghorbani AA (2009) A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE symposium on computational intelligence for security and defense applications. Institute of Electrical & Electronics Engineers (IEEE). https://doi.org/10.1109/cisda.2009.5356528
Traffic data from kyoto university’s honeypots. http://www.takakura.com/Kyoto_data/data/
Vapnik VN (2000) The nature of statistical learning theory. Springer, New York. https://doi.org/10.1007/978-1-4757-3264-1
Xian JQ, Lang FH, Tang XL (2005) A novel intrusion detection method based on clonal selection clustering algorithm. In: 2005 International conference on machine learning and cybernetics, vol 6, pp 3905–3910. IEEE. https://doi.org/10.1109/icmlc.2005.1527620
Yan Q, Yu J (2006) Ainids: an immune-based network intrusion detection system. In: Defense and security symposium, pp 62410U–62410U. International Society for Optics and Photonics
Yang H, Li T, Hu X, Wang F, Zou Y (2014) A survey of artificial immune system based intrusion detection. Sci World J 2014:1–11. https://doi.org/10.1155/2014/156790
Yasir H, Balasaraswathi VR, Journaux L, Sugumaran M (2018) Benchmark datasets for network intrusion detection: a review. Int J Netw Secur 20:645–654
Yin C, Ma L, Feng L (2015) Towards accurate intrusion detection based on improved clonal selection algorithm. Multimed Tools Appl 76:1–14. https://doi.org/10.1007/s11042-015-3117-0
Yin C, Ma L, Feng L (2016) A feature selection method for improved clonal algorithm towards intrusion detection. Int J Pattern Recognit Artif Intell 30(05):1659013
Zargari S, Voorhis D (2012) Feature selection in the corrected KDD-dataset. In: 2012 Third international conference on emerging intelligent data and web technologies. IEEE. https://doi.org/10.1109/eidwt.2012.10
Zhang L, ying BAI Z, long LU Y, xing ZHA Y, wen LI Z (2014) Integrated intrusion detection model based on artificial immune. J China Univ Posts Telecommun 21(2):83–90
Zhao X, Wang G, Li Z (2016) Unsupervised network anomaly detection based on abnormality weights and subspace clustering. In: 2016 Sixth international conference on information science and technology (ICIST). IEEE. https://doi.org/10.1109/icist.2016.7483462
Zhu X (2005) Semi-supervised learning literature survey. Technical Report 1530, Department of Computer Sciences, University of Wosconsin, Madison
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Belhadj aissa, N., Guerroumi, M. & Derhab, A. NSNAD: negative selection-based network anomaly detection approach with relevant feature subset. Neural Comput & Applic 32, 3475–3501 (2020). https://doi.org/10.1007/s00521-019-04396-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-019-04396-2