Abstract
In the field of network security, the detection of possible intrusions is an important task to prevent and analyse attacks. Machine learning has been adopted as a particular supporting technique over the last years. However, the majority of related published work uses post mortem log files and fails to address the required real-time capabilities of network data feature extraction and machine learning based analysis [1,2,3,4,5]. We introduce the network feature extractor library FEX, which is designed to allow real-time feature extraction of network data. This library incorporates 83 statistical features based on reassembled data flows. The introduced Cython implementation allows processing individual packets within 4.58 μs. Based on the features extracted by FEX, existing intrusion detection machine learning models were examined with respect to their real-time capabilities. An identified Decision-Tree Classifier model was thus further optimised by transpiling it into C Code. This reduced the prediction time of a single sample to 3.96 μs on average. Based on the feature extractor and the improved machine learning model an IDS system was implemented which supports a data throughput between 63.7 Mbit/s and 2.5 Gbit/s making it a suitable candidate for a real-time, machine-learning based IDS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Santiago, S., Castro e Silva, J., Maia, J.: NTCS: a real time flow-based network traffic classification system. In: Proceedings of the 10th International Conference on Network and Service Management, CNSM 2014, pp. 368–371 (2015)
Sangkatsanee, P., Wattanapongsakorn, N., Charnsripinyo, C.: Practical real-time intrusion detection using machine learning approaches. Comput. Commun. 34, 2227–2235 (2011). https://doi.org/10.1016/j.comcom.2011.07.001
Rathore, M.M., Ahmad, A., Paul, A.: Real time intrusion detection system for ultra-high-speed big data environments. J. Supercomput. 72(9), 3489–3510 (2016). https://doi.org/10.1007/s11227-015-1615-5
Kim, A., Park, M., Lee, D.H.: AI-IDS: application of deep learning to real-time web intrusion detection. IEEE Access 8, 70245–70261 (2020)
Canadian Institute for Cybersecurity: Canadian Institute for Cybersecurity - Applications - CICFlowMeter (formerly ISCXFlowMeter). https://www.unb.ca/cic/research/applications.html
Mitchell, R., Chen, I.-R.: A survey of intrusion detection in wireless network applications. Comput. Commun. 42, 1–23 (2014)
Bhuyan, M.H., Bhattacharyya, D.K., Kalita, J.K.: Network traffic anomaly detection techniques and systems. In: Network Traffic Anomaly Detection and Prevention. CCN, pp. 115–169. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65188-0_4
Lashkari, A.H., Gil, G.D., Mamun, M.S.I., Ghorbani, A.A.: Characterization of encrypted and VPN traffic using time-related features. In: Proceedings of the 2nd International Conference on Information Systems Security and Privacy - Volume 1: INSTICC, pp. 407–414. SciTePress (2016)
Lashkari, A.H., Gil, G.D., Mamun, M.S.I., Ghorbani, A.A.: Characterization of TOR traffic using time based features. In: Proceedings of the 3rd International Conference on Information Systems Security and Privacy - Volume 1 INSTICC, pp. 253–262. SciTePress (2017)
Canadian Institute for Cybersecurity: Intrusion Detection Evaluation Dataset (CICIDS2017). https://www.unb.ca/cic/datasets/ids-2017.html
West, D.H.D.: Updating mean and variance estimates: an improved method. Commun. ACM 22(9), 532–535 (1979)
Panigrahi, I.R., Borah, S.: A detailed analysis of cicids2017 dataset for designing intrusion detection systems. J. Eng. Technol. 7, 479–482 (2018)
Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Halford, M.: Creme - online machine learning in python. https://github.com/creme-ml/creme.
Geron, A.: Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 2nd edn. O'Reilly UK Ltd, Newton
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Schaad, A., Binder, D. (2021). FEX – A Feature Extractor for Real-Time IDS. In: Liu, J.K., Katsikas, S., Meng, W., Susilo, W., Intan, R. (eds) Information Security. ISC 2021. Lecture Notes in Computer Science(), vol 13118. Springer, Cham. https://doi.org/10.1007/978-3-030-91356-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-91356-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91355-7
Online ISBN: 978-3-030-91356-4
eBook Packages: Computer ScienceComputer Science (R0)