Skip to main content
SpringerLink
Log in

FEX – A Feature Extractor for Real-Time IDS

  • Conference paper
  • First Online:
Information Security (ISC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 13118))

Included in the following conference series:

  • 965 Accesses

Abstract

In the field of network security, the detection of possible intrusions is an important task to prevent and analyse attacks. Machine learning has been adopted as a particular supporting technique over the last years. However, the majority of related published work uses post mortem log files and fails to address the required real-time capabilities of network data feature extraction and machine learning based analysis [1,2,3,4,5]. We introduce the network feature extractor library FEX, which is designed to allow real-time feature extraction of network data. This library incorporates 83 statistical features based on reassembled data flows. The introduced Cython implementation allows processing individual packets within 4.58 μs. Based on the features extracted by FEX, existing intrusion detection machine learning models were examined with respect to their real-time capabilities. An identified Decision-Tree Classifier model was thus further optimised by transpiling it into C Code. This reduced the prediction time of a single sample to 3.96 μs on average. Based on the feature extractor and the improved machine learning model an IDS system was implemented which supports a data throughput between 63.7 Mbit/s and 2.5 Gbit/s making it a suitable candidate for a real-time, machine-learning based IDS.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Santiago, S., Castro e Silva, J., Maia, J.: NTCS: a real time flow-based network traffic classification system. In: Proceedings of the 10th International Conference on Network and Service Management, CNSM 2014, pp. 368–371 (2015)

    Google Scholar 

  2. Sangkatsanee, P., Wattanapongsakorn, N., Charnsripinyo, C.: Practical real-time intrusion detection using machine learning approaches. Comput. Commun. 34, 2227–2235 (2011). https://doi.org/10.1016/j.comcom.2011.07.001

    Article  Google Scholar 

  3. Rathore, M.M., Ahmad, A., Paul, A.: Real time intrusion detection system for ultra-high-speed big data environments. J. Supercomput. 72(9), 3489–3510 (2016). https://doi.org/10.1007/s11227-015-1615-5

    Article  Google Scholar 

  4. Kim, A., Park, M., Lee, D.H.: AI-IDS: application of deep learning to real-time web intrusion detection. IEEE Access 8, 70245–70261 (2020)

    Article  Google Scholar 

  5. Canadian Institute for Cybersecurity: Canadian Institute for Cybersecurity - Applications - CICFlowMeter (formerly ISCXFlowMeter). https://www.unb.ca/cic/research/applications.html

  6. Mitchell, R., Chen, I.-R.: A survey of intrusion detection in wireless network applications. Comput. Commun. 42, 1–23 (2014)

    Article  Google Scholar 

  7. Bhuyan, M.H., Bhattacharyya, D.K., Kalita, J.K.: Network traffic anomaly detection techniques and systems. In: Network Traffic Anomaly Detection and Prevention. CCN, pp. 115–169. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65188-0_4

    Chapter  Google Scholar 

  8. Lashkari, A.H., Gil, G.D., Mamun, M.S.I., Ghorbani, A.A.: Characterization of encrypted and VPN traffic using time-related features. In: Proceedings of the 2nd International Conference on Information Systems Security and Privacy - Volume 1: INSTICC, pp. 407–414. SciTePress (2016)

    Google Scholar 

  9. Lashkari, A.H., Gil, G.D., Mamun, M.S.I., Ghorbani, A.A.: Characterization of TOR traffic using time based features. In: Proceedings of the 3rd International Conference on Information Systems Security and Privacy - Volume 1 INSTICC, pp. 253–262. SciTePress (2017)

    Google Scholar 

  10. Canadian Institute for Cybersecurity: Intrusion Detection Evaluation Dataset (CICIDS2017). https://www.unb.ca/cic/datasets/ids-2017.html

  11. https://github.com/dobinder/FEX

  12. West, D.H.D.: Updating mean and variance estimates: an improved method. Commun. ACM 22(9), 532–535 (1979)

    Article  Google Scholar 

  13. Panigrahi, I.R., Borah, S.: A detailed analysis of cicids2017 dataset for designing intrusion detection systems. J. Eng. Technol. 7, 479–482 (2018)

    Google Scholar 

  14. Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  15. Halford, M.: Creme - online machine learning in python. https://github.com/creme-ml/creme.

  16. Geron, A.: Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 2nd edn. O'Reilly UK Ltd, Newton

    Google Scholar 

  17. https://github.com/BayesWitnesses/m2cgen/

Download references

Author information

Authors and Affiliations

  1. Offenburg University of Applied Sciences, Badstrasse 24, 77652, Offenburg, Germany

    Andreas Schaad & Dominik Binder

Authors
  1. Andreas Schaad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominik Binder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Andreas Schaad or Dominik Binder .

Editor information

Editors and Affiliations

  1. Monash University, Clayton, VIC, Australia

    Joseph K. Liu

  2. Norwegian University of Science and Technology, Gjøvik, Norway

    Sokratis Katsikas

  3. Technical University of Denmark, Kongens Lyngby, Denmark

    Weizhi Meng

  4. University of Wollongong, Wollongong, NSW, Australia

    Willy Susilo

  5. Petra Christian University, Surabaya, Indonesia

    Rolly Intan

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schaad, A., Binder, D. (2021). FEX – A Feature Extractor for Real-Time IDS. In: Liu, J.K., Katsikas, S., Meng, W., Susilo, W., Intan, R. (eds) Information Security. ISC 2021. Lecture Notes in Computer Science(), vol 13118. Springer, Cham. https://doi.org/10.1007/978-3-030-91356-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91356-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91355-7

  • Online ISBN: 978-3-030-91356-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation