Skip to main content
Springer Nature Link
Log in

Unsupervised anomaly detection for network traffic using artificial immune network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In the existing approaches of multifarious knowledge based anomaly detection for network traffic, the priori knowledge labelled by human experts has to be consecutively updated for identification of new anomalies. Because anomalies usually show different patterns from the majority of network activities, it is hard to detect them based on the priori knowledge. Unsupervised anomaly detection using autonomous techniques without any priori knowledge is an effective strategy to overcome this drawback. In this paper, we propose a novel model of Unsupervised Anomaly Detection approach based on Artificial Immune Network (UADAIN) that consists of unsupervised clustering, cluster partition and anomaly detection. Our model uses the aiNet based unsupervised clustering approach to generate cluster centroids from network traffic, and the Cluster Centroids based Partition algorithm (CCP) then coarsely partition cluster centroids in the training phase as the self set (normal rules) and antibody set (anomalous rules). In test phase, to keep consecutive evolution of selves and antibodies, we introduce the Immune Network based Anomaly Detection model (INAD) to automatically learn and evolve the self set and antibody set. To evaluate the effectiveness of UADAIN, we conduct simulation experiments on ISCX 2012 IDS dataset and NSL-KDD dataset. In comparison with two popular anomaly detection approaches based on K-means clustering and aiNet-HC clustering, respectively, the experiment results demonstrate that UADAIN achieves better detection performance in detecting anomalies of network traffic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  1. Leung K, Leckie C (2005) Unsupervised anomaly detection in network intrusion detection using clusters. In: Proceedings of the Twenty-eighth Australasian conference on Computer Science Vol 38, pp 333–342. Australian Computer Society, Inc

  2. Tan Z, Jamdagni A, He X, Nanda P, Liu RP, Jiankun H (2015) Detection of denial-of-service attacks based on computer vision techniques. IEEE Trans Comput 64(9):2519–2533

    Article  MathSciNet  Google Scholar 

  3. Garg S, Kaur K, Kumar N, Rodrigues JJPC (2019) Hybrid deep learning-based anomaly detection scheme for suspicious flow detection in sdn: a social multimedia perspective. IEEE Trans Multimed 21(3):566–578

    Article  Google Scholar 

  4. Shon T, Moon J (2007) A hybrid machine learning approach to network anomaly detection. Inf Sci 177(18):3799–3821

    Article  Google Scholar 

  5. Tolga E, Serdar KS (2020) Unsupervised anomaly detection with lstm neural networks. IEEE Trans Neural Netw Learn Syst 31(8):3127–3141

    Article  MathSciNet  Google Scholar 

  6. Garg S, Kaur K, Kumar N, Kaddoum G, Zomaya AY, Rajiv R (2019) A hybrid deep learning-based model for anomaly detection in cloud datacenter networks. IEEE Trans Netw Serv Manage 16(3):924–935

    Article  Google Scholar 

  7. Anderson HH, Luiz FC, Lucas DHS, Taufik A, Proenca ML Jr (2018) Network anomaly detection system using genetic algorithm and fuzzy logic. Expert Syst Appl 92:390–402

    Article  Google Scholar 

  8. Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):15

    Article  Google Scholar 

  9. Jadidi Z, Muthukkumarasamy V, Sithirasenan E, Singh K (2015) Flow-based anomaly detection using semisupervised learning. In: Signal processing and communication systems (ICSPCS), 2015 9th international conference on, IEEE. pp 1–5

  10. Bhuyan MH, Bhattacharyya DK, Kalita JK (2014) Towards an unsupervised method for network anomaly detection in large datasets. Comput Inf 33(1):1–34

    Google Scholar 

  11. Gogoi P, Borah B, Bhattacharyya DK (2010) Anomaly detection analysis of intrusion data using supervised and unsupervised approach. J Converg Inf Technol 5(1):95–110

    Google Scholar 

  12. Mazel J, Casas P, Fontugne R, Fukuda K, Owezarski P (2015) Hunting attacks in the dark: clustering and correlation analysis for unsupervised anomaly detection. Int J Netw Manage 25(5):283–305

    Article  Google Scholar 

  13. Mazel J (2011)Unsupervised network anomaly detection. Thesis

  14. Casas P, Mazel J, Owezarski P (2011) Unada: unsupervised network anomaly detection using sub-space outliers ranking. International conference on research in networking. Springer, Berlin, pp 40–51

    Google Scholar 

  15. Portnoy L, Eskin E, Stolfo S (2001)Intrusion detection with unlabeled data using clustering. In: In Proceedings of ACM CSS workshop on data mining applied to security (DMSA-2001)

  16. Eskin E, Arnold A, Prerau M, Portnoy L, Stolfo S (2002)A geometric framework for unsupervised anomaly detection, pp 77–101. Springer, Berlin

  17. Mnz G, Li S, Carle G (2007) Traffic anomaly detection using k-means clustering. GI/ITG Workshop MMBnet

  18. Fang L, Le-Ping L (2005) Unsupervised anomaly detection based n an evolutionary artificial immune network. Workshops on applications of evolutionary computation. Springer, Berlin, pp 166–174

    Google Scholar 

  19. Dromard J, Roudiere G, Owezarski P (2016) Online and scalable unsupervised network anomaly detection method. IEEE Trans Netw Serv Manage 14(1):34–47

    Article  Google Scholar 

  20. Lau H, Timmis J, Bate I (2009) Anomaly detection inspired by immune network theory: a proposal. In: 2009 IEEE congress on evolutionary computation, pp 3045–3051. IEEE

  21. Li K-L, Huang H-K, Tian S-F, Xu W (2003) Improving one-class svm for anomaly detection. In: Machine learning and cybernetics, 2003 international conference on, vol 5, pp 3077–3081. IEEE

  22. Ippoliti D, Jiang C, Ding Z, Zhou X (2016) Online adaptive anomaly detection for augmented network flows. ACM Trans Autonom Adapt Syst (TAAS) 11(3):17

    Google Scholar 

  23. Shyu M-L, Chen S-C, Sarinnapakorn K, Chang LW (2003) A novel anomaly detection scheme based on principal component classifier. Report, DTIC Document

  24. Lakhina A, Crovella M, Diot C (2005) Mining anomalies using traffic feature distributions. ACM SIGCOMM Comput Commun Rev 35:217–228

    Article  Google Scholar 

  25. Huang L, Nguyen XL, Garofalakis M, Jordan MI, Joseph A, Taft N (2006) In-network pca and anomaly detection. In: NIPS, pp 617–624

  26. Syarif I, Prugel-Bennett A, Wills G (2012) Unsupervised clustering approach for network anomaly detection. International conference on networked digital technologies. Springer, Berlin, pp 135–145

    Chapter  Google Scholar 

  27. Zanero S, Savaresi SM (2004) Unsupervised learning techniques for an intrusion detection system. In: Proceedings of the 2004 ACM symposium on applied computing, pp 412–419. ACM

  28. Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. ACM SIGKDD Explor Newsl 6(1):90–105

    Article  Google Scholar 

  29. Casas P, Mazel J, Owezarski P (2012) Unsupervised network intrusion detection systems: detecting the unknown without knowledge. Comput Commun 35(7):772–783

    Article  Google Scholar 

  30. Dromard J, Roudire G, Owezarski P (2015) Unsupervised network anomaly detection in real-time on big data. In: East European conference on advances in databases and information systems, pp 197–206. Springer, Berlin

  31. Yang C, Deng F, Yang H (2007) An unsupervised anomaly detection approach using subtractive clustering and hidden markov model. In: Communications and networking in China, 2007. CHINACOM’07. Second International Conference on, pp 313–316. IEEE

  32. Leon E, Nasraoui O, Gomez J (2004) Anomaly detection based on unsupervised niche clustering with application to network intrusion detection. In: Evolutionary Computation, 2004. CEC2004. Congress on, vol  1, pp 502–508. IEEE

  33. de Castro LN, von Zuben FJ (2001) ainet: an artificial immune network for data analysis. Data Min Heuristic Approach 2001(1):231–259

  34. Timmis J, Hone A, Stibor T, Clark E (2008) Theoretical advances in artificial immune systems. Theoret Comput Sci 403(1):11–32

    Article  MathSciNet  Google Scholar 

  35. Duma M, Twala B (2019) Sparseness reduction in collaborative filtering using a nearest neighbour artificial immune system with genetic algorithms. Expert Syst Appl 132:110–125

    Article  Google Scholar 

  36. Dasgupta D, Yu S, Majumdar NS (2005) Milacmultilevel immune learning algorithm and its application to anomaly detection. Soft Comput 9(3):172–184

    Article  Google Scholar 

  37. Seredynski F, Bouvry P (2007) Anomaly detection in tcp/ip networks using immune systems paradigm. Comput Commun 30(4):740–749

    Article  Google Scholar 

  38. Li D, Liu S, Zhang H (2015) A negative selection algorithm with online adaptive learning under small samples for anomaly detection. Neurocomputing 149(B):515–525

    Article  Google Scholar 

  39. Shi YQ, Li R, Peng X, Yue G (2016) Network security situation prediction approach based on clonal selection and scgm(1 1)c model. J Int Technol 17(3):421–429

    Google Scholar 

  40. Bo Y, Meifang Y (2021) Data-driven network layer security detection model and simulation for the internet of things based on an artificial immune system. Neural Comput Appl 33(2):655–666

    Article  Google Scholar 

  41. Qian S, Ye Y, Jiang B, Wang J (2016) Constrained multiobjective optimization algorithm based on immune system model. IEEE Trans Cybern 46(9):2056–2069

    Article  Google Scholar 

  42. Shi YQ, Li R, Zhang Y, Peng X (2015) An immunity-based time series prediction approach and its application for network security situation. Intell Serv Robot 8(1):1–22

    Article  Google Scholar 

  43. Dudek G (2017) Artificial immune system with local feature selection for short-term load forecasting. IEEE Trans Evol Comput 21(1):116–130

    Article  Google Scholar 

  44. Li T (2005) An immunity based network security risk estimation. Sci China Ser F Inf Sci 48(5):557–578

    Article  MathSciNet  Google Scholar 

  45. Alizadeh E, Meskin N, Khorasani K (2016) A negative selection immune system inspired methodology for fault diagnosis of wind turbines. IEEE Trans Cybern 47(11):3799–3813

    Article  Google Scholar 

  46. Jerne NK (1974) Towards a network theory of the immune system. Annales d’immunologie 125:373–389

    Google Scholar 

  47. Rassam MA, Maarof MA (2012) Artificial immune network clustering approach for anomaly intrusion detection. J Adv Inf Technol 3(3):147–154

    Google Scholar 

  48. Shiravi A, Shiravi H, Tavallaee M, Ghorbani AA (2012) Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput Security 31(3):357–374

    Article  Google Scholar 

  49. Tavallaee M, Bagheri E, Lu W, Ghorbani A-A (2009) A detailed analysis of the kdd cup 99 data set. 2009 IEEE symposium on computational intelligence for security and defense applications, pp 1–6

  50. Sheikhan M, Jadidi Z (2014) Flow-based anomaly detection in high-speed links using modified gsa-optimized neural network. Neural Comput Appl 24(3–4):599–611

    Article  Google Scholar 

  51. Li W, Canini M, Moore AW, Bolla R (2009) Efficient application identification and the temporal and spatial stability of classification schema. Comput Netw 53(6):790–809

    Article  Google Scholar 

  52. Iglesias F, Zseby T (2015) Analysis of network traffic features for anomaly detection. Mach Learn 101(1–3):59–84

    Article  MathSciNet  Google Scholar 

  53. Maloof MA (2005) Machine learning and data mining for computer security: methods and applications. pp 23–45. Springer-Verlag, New York

Download references

Acknowledgements

This study was funded by the National Natural Science Foundation of China under Grant No. 62172182, China Scholarship Council, Australian Research Council Discovery Project DP150104871, Hunan Provincial Natural Science Foundation of China under Grant No. 2020JJ4490, the Scientific Research Fund of Hunan Provincial Education Department of China under Grant No.18A449.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Huaihua University, Huaihua, 418000, China

    Yuanquan Shi

  2. School of Computer Science and Engineering, Sun Yat-Sen University, Guangzhou, 510006, China

    Hong Shen

  3. School of Computer Science, The University of Adelaide, SA, 5005, Australia

    Yuanquan Shi & Hong Shen

Authors
  1. Yuanquan Shi
    View author publications

    Search author on:PubMed Google Scholar

  2. Hong Shen
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Yuanquan Shi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Shen, H. Unsupervised anomaly detection for network traffic using artificial immune network. Neural Comput & Applic 34, 13007–13027 (2022). https://doi.org/10.1007/s00521-022-07156-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07156-x

Keywords