Skip to main content
SpringerLink
Log in
Book cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2022: Machine Learning and Principles and Practice of Knowledge Discovery in Databases pp 186–201Cite as

Towards a General Model for Intrusion Detection: An Exploratory Study

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1753))

Abstract

Exercising Machine Learning (ML) algorithms to detect intrusions is nowadays the de-facto standard for data-driven detection tasks. This activity requires the expertise of the researchers, practitioners, or employees of companies that also have to gather labeled data to learn and evaluate the model that will then be deployed into a specific system. Reducing the expertise and time required to craft intrusion detectors is a tough challenge, which in turn will have an enormous beneficial impact in the domain. This paper conducts an exploratory study that aims at understanding to which extent it is possible to build an intrusion detector that is general enough to learn the model once and then be applied to different systems with minimal to no effort. Therefore, we recap the issues that may prevent building general detectors and propose software architectures that have the potential to overcome them. Then, we perform an experimental evaluation using several binary ML classifiers and a total of 16 feature learners on 4 public attack datasets. Results show that a model learned on a dataset or a system does not generalize well as is to other datasets or systems, showing poor detection performance. Instead, building a unique model that is then tailored to a specific dataset or system may achieve good classification performance, requiring less data and far less expertise from the final user.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Sommer, R., Paxson, V.: Outside the closed world: on using machine learning for network intrusion detection. In: 2010 IEEE Symposium on Security and Privacy, pp. 305–316. IEEE, May 2010

    Google Scholar 

  2. Catillo, M., Del Vecchio, A., Pecchia, A., Villano, U.: Transferability of machine learning models learned from public intrusion detection datasets: the CICIDS2017 case study. Softw. Qual. J. 30, 955–981 (2022). https://doi.org/10.1007/s11219-022-09587-0

    Article  Google Scholar 

  3. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  4. Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., Madry, A.: Adversarially robust generalization requires more data. In: Advances in Neural Information Processing Systems, vol. 31 (2018). Accessed 07 Apr 2022

    Google Scholar 

  5. Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting batch normalization for practical domain adaptation, November 2016. http://arxiv.org/abs/1603.04779. Accessed 07 Apr 2022

  6. Jindal, I., Nokleby, M., Chen, X.: Learning deep networks from noisy labels with dropout regularization. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 967–972, December 2016. https://doi.org/10.1109/ICDM.2016.0121

  7. Chen, X.W., Lin, X.: Big data deep learning: challenges and perspectives. IEEE Access 2, 514–525 (2014)

    Article  Google Scholar 

  8. Lawrence, S., Giles, C.L.: Overfitting and neural networks: conjugate gradient and backpropagation. In: Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, vol. 1, pp. 114–119. IEEE, July 2000

    Google Scholar 

  9. Song, H., et al.: Learning from noisy labels with deep neural networks: a survey. IEEE Trans. Neural Netw. Learn. Syst. (2022, article in press). https://doi.org/10.1109/TNNLS.2022.3152527

  10. Krogh, A., Hertz, J.: A simple weight decay can improve generalization. In: Advances in Neural Information Processing Systems, vol. 4 (1991)

    Google Scholar 

  11. Caruana, R., Lawrence, S., Giles, C.: Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Advances in Neural Information Processing Systems, vol. 13 (2000)

    Google Scholar 

  12. Prechelt, L.: Early stopping - but when? In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the trade. LNCS, vol. 1524, pp. 55–69. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49430-8_3

    Chapter  Google Scholar 

  13. Sietsma, J., Dow, R.J.: Creating artificial neural networks that generalize. Neural Netw. 4(1), 67–79 (1991)

    Article  Google Scholar 

  14. Kawaguchi, K., Kaelbling, L.P., Bengio, Y.: Generalization in deep learning. arXiv preprint arXiv:1710.05468 (2017)

  15. Cestnik, B., Bratko, I.: On estimating probabilities in tree pruning. In: Kodratoff, Y. (ed.) Machine Learning — EWSL-91: European Working Session on Learning Porto, Portugal, March 6–8, 1991 Proceedings, pp. 138–150. Springer, Heidelberg (1991). https://doi.org/10.1007/BFb0017010

    Chapter  Google Scholar 

  16. Gao, S.H., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.: Res2Net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 43(2), 652–662 (2019)

    Article  Google Scholar 

  17. Bishop, C.: Pattern Recognition and Machine Learning. Springer, Berlin (2006). ISBN: 0-387-31073-8

    Google Scholar 

  18. Rivolli, A., Garcia, L.P., Soares, C., Vanschoren, J., de Carvalho, A.C.: Meta-features for meta-learning. Knowl.-Based Sys. 240, 108101 (2022)

    Google Scholar 

  19. Cotroneo, D., Natella, R., Rosiello, S.: A fault correlation approach to detect performance anomalies in Virtual Network Function chains. In: 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pp. 90–100. IEEE (2017)

    Google Scholar 

  20. Zoppi, T., Ceccarelli, A., Bondavalli, A.: MADneSs: a multi-layer anomaly detection framework for complex dynamic systems. IEEE Trans. Dependable Secure Comput. 18(2), 796–809 (2019)

    Article  Google Scholar 

  21. Murtaza, S.S., et al.: A host-based anomaly detection approach by representing system calls as states of kernel modules. In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). IEEE (2013)

    Google Scholar 

  22. Wang, G., Zhang, L., Xu, W.: What can we learn from four years of data center hardware failures? In: 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 25–36. IEEE, June 2017

    Google Scholar 

  23. Li, Z., Zou, D., Xu, S., Jin, H., Zhu, Y., Chen, Z.: SySeVR: a framework for using deep learning to detect software vulnerabilities. IEEE Trans. Dependable Secure Comput. 19(4), 2244–2258 (2022)

    Article  Google Scholar 

  24. Robles-Velasco, A., Cortés, P., Muñuzuri, J., Onieva, L.: Prediction of pipe failures in water supply networks using logistic regression and support vector classification. Reliab. Eng. Syst. Saf. 196, 106754 (2020)

    Article  Google Scholar 

  25. Ardagna, C., Corbiaux, S., Sfakianakis, A., Douliger, C.: ENISA Threat Landscape 2021. https://www.enisa.europa.eu/topics/threat-risk-management/threats-and-trends. Accessed 6 May 2022

  26. Connell, B.: 2022 SonicWall Threat Report. https://www.sonicwall.com/2022-cyber-threat-report/. Accessed 6 May 2022

  27. Džeroski, S., Ženko, B.: Is combining classifiers with stacking better than selecting the best one? Mach. Learn. 54(3), 255–273 (2004). https://doi.org/10.1023/B:MACH.0000015881.36452.6e

    Article  MATH  Google Scholar 

  28. Khraisat, A., Gondal, I., Vamplew, P., Kamruzzaman, J.: Survey of intrusion detection systems: techniques, datasets, and challenges. Cybersecurity 2(1) (2019). Article number: 20. https://doi.org/10.1186/s42400-019-0038-7

  29. Ring, M., Wunderlich, S., Scheuring, D., Landes, D., Hotho, A.: A survey of network-based intrusion detection data sets. Comput. Secur. 86, 147–167 (2019)

    Article  Google Scholar 

  30. Elsayed, M.S., Le-Khac, N.A., Jurcut, A.D.: InSDN: a novel SDN intrusion dataset. IEEE Access 8, 165263–165284 (2020)

    Article  Google Scholar 

  31. Sharafaldin, I., et al.: Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: ICISSP, pp. 108–116, January 2018

    Google Scholar 

  32. Lashkari, A.H., et al.: Toward developing a systematic approach to generate benchmark Android malware datasets and classification. In: 2018 International Carnahan Conference on Security Technology (ICCST), pp. 1–7. IEEE, October 2018

    Google Scholar 

  33. Resende, P.A.A., Drummond, A.C.: A survey of random forest based methods for intrusion detection systems. ACM Comput. Surv. (CSUR) 51(3), 1–36 (2018)

    Article  Google Scholar 

  34. Shwartz-Ziv, R., Armon, A.: Tabular data: deep learning is not all you need. Inf. Fusion 81, 84–90 (2022)

    Article  Google Scholar 

  35. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  36. Chen, T., et al.: XGBoost: eXtreme gradient boosting. R Package Version 0.4-2, 1(4), 1–4 (2015)

    Google Scholar 

  37. Howard, J., Gugger, S.: Fastai: a layered API for deep learning. Information 11(2), 108 (2020)

    Article  Google Scholar 

  38. Zhao, Y., Nasrullah, Z., Li, Z.: PyOD: a python toolbox for scalable outlier detection. arXiv preprint arXiv:1901.01588 (2019)

  39. Buitinck, L., et al.: API design for machine learning software: experiences from the scikit-learn project. arXiv preprint arXiv:1309.0238 (2013)

  40. Luque, A., et al.: The impact of class imbalance in classification performance metrics based on the binary confusion matrix. Pattern Recogn. 91, 216–231 (2019)

    Article  Google Scholar 

  41. Ucci, D., Aniello, L., Baldoni, R.: Survey of machine learning techniques for malware analysis. Comput. Secur. 81, 123–147 (2019)

    Article  Google Scholar 

  42. Demetrio, L., et al.: Adversarial exemples: a survey and experimental evaluation of practical attacks on machine learning for windows malware detection. ACM Trans. Priv. Secur. (TOPS) 24(4), 1–31 (2021)

    Article  Google Scholar 

  43. Zhauniarovich, Y., Khalil, I., Yu, T., Dacier, M.: A survey on malicious domains detection through DNS data analysis. ACM Comput. Surv. (CSUR) 51(4), 1–36 (2018)

    Article  Google Scholar 

  44. Oliveira, R.A., Raga, M.M., Laranjeiro, N., Vieira, M.: An approach for benchmarking the security of web service frameworks. Future Gener. Comput. Syst. 110, 833–848 (2020)

    Article  Google Scholar 

  45. Andresini, G., Appice, A., Malerba, D.: Autoencoder-based deep metric learning for network intrusion detection. Inf. Sci. 569, 706–727 (2021)

    Article  Google Scholar 

  46. Apruzzese, G., Colajanni, M., Ferretti, L., Guido, A., Marchetti, M.: On the effectiveness of machine and deep learning for cyber security. In: 2018 10th International Conference on Cyber Conflict (CyCon), pp. 371–390. IEEE, May 2018

    Google Scholar 

  47. Folino, F., et al.: On learning effective ensembles of deep neural networks for intrusion detection. Inf. Fusion 72, 48–69 (2021)

    Article  Google Scholar 

  48. Arp, D., et al.: Dos and don’ts of machine learning in computer security. In: Proceedings of the USENIX Security Symposium, August 2022

    Google Scholar 

  49. Verkerken, M., D’hooge, L., Wauters, T., Volckaert, B., De Turck, F.: Towards model generalization for intrusion detection: unsupervised machine learning techniques. J. Netw. Syst. Manag. 30 (2022). Article number: 12. https://doi.org/10.1007/s10922-021-09615-7

  50. Haider, W., Hu, J., Slay, J., Turnbull, B.P., Xie, Y.: Generating realistic intrusion detection system dataset based on fuzzy qualitative modeling. J. Netw. Comput. Appl. 87, 185–192 (2017)

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the H2020 Marie Sklodowska-Curie g.a. 823788 (ADVANCE), by the Regione Toscana POR FESR 2014–2020 SPaCe, and by the NextGenerationEU programme, Italian DM737 – CUP B15F21005410003.

Author information

Authors and Affiliations

  1. Department of Mathematics and Informatics, University of Florence, Viale Morgagni 65, 50142, Florence, Italy

    Tommaso Zoppi, Andrea Ceccarelli & Andrea Bondavalli

Authors
  1. Tommaso Zoppi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Ceccarelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Bondavalli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tommaso Zoppi .

Editor information

Editors and Affiliations

  1. University of Sydney, Sydney, Australia

    Irena Koprinska

  2. University of Bari Aldo Moro, Bari, Italy

    Paolo Mignone

  3. University of Pisa, Pisa, Italy

    Riccardo Guidotti

  4. Warsaw University of Technology, Warsaw, Poland

    Szymon Jaroszewicz

  5. Heidelberg University, Heidelberg, Germany

    Holger Fröning

  6. UniCredit, Rome, Italy

    Francesco Gullo

  7. University of Lisbon, Lisbon, Portugal

    Pedro M. Ferreira

  8. Roche, Basel, Switzerland

    Damian Roqueiro

  9. Barcelona Supercomputing Center, Barcelona, Spain

    Gaia Ceddia

  10. Halmstad University, Halmstad, Sweden

    Slawomir Nowaczyk

  11. University of Porto, Porto, Portugal

    João Gama

  12. University of Porto, Porto, Portugal

    Rita Ribeiro

  13. UPC BarcelonaTech, Barcelona, Spain

    Ricard Gavaldà

  14. University of Naples Federico II, Naples, Italy

    Elio Masciari

  15. University of North Carolina, Charlotte, USA

    Zbigniew Ras

  16. ICAR-CNR, Rende, Italy

    Ettore Ritacco

  17. University of Pisa, Pisa, Italy

    Francesca Naretto

  18. Aalen University of Applied Sciences, Aalen, Germany

    Andreas Theissler

  19. Warsaw University of Technology, Warszaw, Poland

    Przemyslaw Biecek

  20. KU Leuven, Leuven, Belgium

    Wouter Verbeke

  21. University of Duisburg-Essen, Essen, Germany

    Gregor Schiele

  22. Graz University of Technology, Graz, Austria

    Franz Pernkopf

  23. AMD, Dublin, Ireland

    Michaela Blott

  24. UniCredit, Rome, Italy

    Ilaria Bordino

  25. UniCredit, Milan, Italy

    Ivan Luciano Danesi

  26. National Agency for New Technologies, Rome, Italy

    Giovanni Ponti

  27. Unicredit, Rome, Italy

    Lorenzo Severini

  28. University of Bari Aldo Moro, Bari, Italy

    Annalisa Appice

  29. University of Bari Aldo Moro, Bari, Italy

    Giuseppina Andresini

  30. University of Lisbon, Lisbon, Portugal

    Ibéria Medeiros

  31. University of Lisbon, Lisbon, Portugal

    Guilherme Graça

  32. Northwestern University, Chicago, USA

    Lee Cooper

  33. Roche, Basel, Switzerland

    Naghmeh Ghazaleh

  34. University of Lausanne, Lausanne, Switzerland

    Jonas Richiardi

  35. Novartis, Basel, Switzerland

    Diego Saldana

  36. Novartis, Basel, Switzerland

    Konstantinos Sechidis

  37. Fondazione IRCCS Ca’ Granda Ospedale Maggiore Policlinico, Milan, Italy

    Arif Canakoglu

  38. Politecnico di Milano, Milan, Italy

    Sara Pido

  39. Politecnico di Milano, Milan, Italy

    Pietro Pinoli

  40. University of Waikato, Hamilton, New Zealand

    Albert Bifet

  41. Halmstad University, Halmstad, Sweden

    Sepideh Pashami

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zoppi, T., Ceccarelli, A., Bondavalli, A. (2023). Towards a General Model for Intrusion Detection: An Exploratory Study. In: Koprinska, I., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2022. Communications in Computer and Information Science, vol 1753. Springer, Cham. https://doi.org/10.1007/978-3-031-23633-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23633-4_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23632-7

  • Online ISBN: 978-3-031-23633-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation