Abstract
As network traffic bandwidth is increasing at an exponential rate, it’s impossible to keep up with the speed of networks by just increasing the speed of processors. Besides, increasingly complex intrusion detection methods only add further to the pressure on network intrusion detection (NIDS) platforms, so the continuous increasing speed and throughput of network poses new challenges to NIDS. To make NIDS usable in Gigabit Ethernet, the ideal policy is using a load balancer to split the traffic data and forward those to different detection sensors, which can analyze the splitting data in parallel. In order to make each slice contains all the evidence necessary to detect a specific attack, the load balancer design must be complicated and it becomes a new bottleneck of NIDS. To simplify the load balancer this paper put forward a distributed neural network learning algorithm (DNNL). Using DNNL a large data set can be split randomly and each slice of data is presented to an independent neural network; these networks can be trained in distribution and each one in parallel. Completeness analysis shows that DNNL’s learning algorithm is equivalent to training by one neural network which uses the technique of regularization. The experiments to check the completeness and efficiency of DNNL are performed on the KDD’99 Data Set which is a standard intrusion detection benchmark. Compared with other approaches on the same benchmark, DNNL achieves a high detection rate and low false alarm rate.
Similar content being viewed by others
References
Song, H.Y., Lockwood, J.W.: Efficient packet classification for network intrusion detection using FPGA. In: Proceedings of the 13th International Symposium on Field-programmable Gate Arrays, pp. 238–245. Monterey (2005)
Yang W., Fang B.X., Liu B., Zhang H.L.: Intrusion detection system for high-speed network. J. Comput. Commun. 27, 1288–1294 (2004)
Baker, Z.K., Prasanna, V.K.: Automatic synthesis of efficient intrusion detection systems on FPGAs. In: Proceedings of the 14th Field Programmable Logic and Application, pp. 311–321. Leuven, Belgium (2004)
Baker, Z.K., Prasanna, V.K.: A methodology for synthesis of efficient intrusion detection systems on FPGAs. In: Proceedings of the 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), pp. 135–144. Napa (2004)
McAlerney, J., Coit, C., Staniford, S.: Towards faster string matching for intrusion detection or exceeding the speed of snort. In: Proceedings of DARPA Information Survivability Conference and Exposition, pp. 367–373. Anaheim (2001)
Tuck, N., Sherwood, T., Calder, B., Varghese, G.: Deterministic memory-efficient string matching algorithms for intrusion detection. In: Proceedings of the 23rd Conference of the IEEE Communications Society, pp. 2628–2639. Hong Kong (2004)
Tan, L., Sherwood, T.: A high throughput string matching architecture for intrusion detection and prevention. In: Proceedings of the 32nd International Symposium on Computer Architecture, pp. 112–122. Madison, Wisconsin (2005)
Aggarwal C., Yu S.: An effective and efficient algorithm for high-dimensional outlier detection. J. Int. J. Very Large Data Bases 14, 211–221 (2005)
Rawat S., Pujari A.K., Gulati V.P.: On the use of singular value decomposition for a fast intrusion detection system. J. Electronic Notes Theor. Comput. Sci. 142, 215–228 (2006)
Kruegel, C., Valeur, F., Vigna, G., Kemmerer, R.: Stateful intrusion detection for high-speed networks. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 285–294. California (2002)
Lai, H.G., Cai, S.W., Huang, H., Xie, J.Y., Li, H.: A parallel intrusion detection system for high-speed networks. In: Proceedings of Applied Cryptography and Network Security: Second International Conference, pp.439–451. ACNS 2004, Yellow Mountain (2004)
Jiang W.B., Song H., Dai Y.Q.: Real-time intrusion detection for high-speed networks. J. Comput. Secur. 24, 287–294 (2005)
Xinidis K., Charitakis I., Antonatos S., Anagnostakis K.G., Markatos E.P.: An active splitter architecture for intrusion detection and prevention. J. IEEE Trans. Dependable. Secure Comput. 3, 31–44 (2006)
Schaelicke, L., Wheeler, K., Freeland, C.: SPANIDS: a scalable network intrusion detection loadbalancer. In: Proceedings of the 2nd Conference on Computing Frontiers, pp. 315–322. Ischia (2005)
Szalay A., Gray J.: The world-wide telescope. Science 293, 2037–2040 (2001)
Martone M.E., Gupta A., Ellisman M.H.: E-neuroscience: challenges and triumphs in integrating distributed data from molecules to brains. Nature Neurosci. 7, 467–472 (2004)
Wroe C., Goble C., Greenwood M., Lord P., Miles S., Papay J., Payne T., Moreau L.: Automating experiments using semantic data on a bioinformatics grid. IEEE Intell. Syst. 19, 48–55 (2004)
Wang Y.X., Behera S.R., Wong J., Helmer G., Honavar V., Miller L., Lutz R., Slagell M.: Towards the automatic generation of mobile agents for distributed intrusion detection system. J. Syst. Softw. 79, 1–14 (2006)
Bala, J., Weng, Y., Williams, A., Gogia, B.K., Lesser, H.K.: Applications of Distributed Mining Techniques For Knowledge Discovery in Dispersed Sensory Data. In: Proceedings of the 7th Joint Conference on Information Sciences, pp. 1–4. Cary (2003)
Kourai, K., Chiba, S.: HyperSpector virtual distributed monitoring environments for secure intrusion detection. In: Proceedings of the 1st ACM/USENIX International Conference on Virtual Execution Environments, pp. 197–207. Chicago (2005)
Folino, G., Pizzuti, C., Spezzano, G.: GP ensemble for distributed intrusion detection systems. In: Proceedings of the 3rd International Conference on Advanced in Pattern Recognition, pp. 54–62. Bath, UK (2005)
Geman S., Bienenstock E., Doursat R.: Neural networks and the bias/variance dilema. Neural Comput. 4, 1–58 (1992)
Kuo R.J., An Y.L., Wang H.S., Chung W.J.: Integration of self-organizing feature maps neural network and genetic K-means algorithm for market segmentation. J. Expert Syst. Appl. 30, 313–324 (2006)
Carpenter G.A., Milenova B.L., Noeske B.W.: Distributed ARTMAP: a neural network for fast distributed supervised learning. J. Neural Networks 11, 793–813 (1998)
Nair T.M., Zheng C.L., Fink J.L., Stuart R.O., Gribskov M.: Rival penalized competitive learning (RPCL): a topology-determining algorithm for analyzing gene expression data. J. Comput. Biol. Chem. 27, 565–574 (2003)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tian, D., Liu, Y. & Xiang, Y. Large-scale network intrusion detection based on distributed learning algorithm. Int. J. Inf. Secur. 8, 25–35 (2009). https://doi.org/10.1007/s10207-008-0061-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10207-008-0061-2