Skip to main content
SpringerLink
Log in

Pre-trained language model-enhanced conditional generative adversarial networks for intrusion detection

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

As cyber threats continue to evolve, ensuring network security has become increasingly critical. Deep learning-based intrusion detection systems (IDS) are crucial for addressing this issue. However, imbalanced training data and limited feature extraction weaken classification performance for intrusion detection. This paper presents a conditional generative adversarial network (CGAN) enhanced by Bidirectional Encoder Representations from Transformers (BERT), a pre-trained language model, for multi-class intrusion detection. This approach augments minority attack data through CGAN to mitigate class imbalance. BERT with robust feature extraction is embedded into the CGAN discriminator to enhance input–output dependency and improve detection through adversarial training. Experiments show the proposed model outperforms baselines on CSE-CIC-IDS2018, NF-ToN-IoT-V2, and NF-UNSW-NB15-v2 datasets, achieving F1-scores of 98.230%, 98.799%, and 89.007%, respectively, and improving F1-scores over baselines by 1.218%\(-\)13.844% 0.215%\(-\)13.779%, and 2.056%\(-\)22.587%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Notes

  1. https://cyber-edge.com/wp-content/uploads/2022/11/CyberEdge-2022-CDR-Report.pdf

  2. https://www.unb.ca/cic/datasets/ids-2018.html

  3. https://rdm.uq.edu.au/files/a4ad7080-ef9c-11ed-a964-b70596e96ad5

  4. https://rdm.uq.edu.au/files/8c6e2a00-ef9c-11ed-827d-e762de186848

References

  1. Chou D, Jiang M (2021) A Survey on Data-driven Network Intrusion Detection. ACM Comput Surv (CSUR) 54(9):1–36

    Article  Google Scholar 

  2. Kilincer IF, Ertam F, Sengur A (2021) Machine learning methods for cyber security intrusion detection: Datasets and comparative study. Comput Netw 188:107840

    Article  Google Scholar 

  3. Gamage S, Samarabandu J (2020) Deep learning methods in network intrusion detection: A survey and an objective comparison. J Netw Comput Appl 169:102767

    Article  Google Scholar 

  4. Mummadi A, Yadav BMK, Sadhwika R, Shitharth S (2021) An appraisal of cyber-attacks and countermeasures using machine learning algorithms. In International Conference on Artificial Intelligence and Data Science, pages 27–40

  5. Wang H, Gu J, Wang S (2017) An effective intrusion detection framework based on SVM with feature augmentation. Knowl-Based Syst 136:130–139

    Article  Google Scholar 

  6. Koc L, Mazzuchi TA, Sarkani S (2012) A network intrusion detection system based on a Hidden Naïve Bayes multiclass classifier. Expert Syst Appl 39(18):13492–13500

    Article  Google Scholar 

  7. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: Synthetic Minority Over-sampling Technique. J Artif Intell Res 16:321–357

    Article  Google Scholar 

  8. Jia H, Liu J, Zhang M, He X, Sun W (2021) Network intrusion detection based on IE-DBN model. Comput Commun 178:131–140

    Article  Google Scholar 

  9. Wu T, Fan H, Zhu H, You C, Zhou H (2022) Huang X (2022) Intrusion detection system combined enhanced random forest with smote algorithm. EURASIP J Adv Signal Process 1:1–20

    Google Scholar 

  10. Mikhail JW, Fossaceca JM, Iammartino R (2019) A semi-boosted nested model with sensitivity-based weighted binarization for multi-domain network intrusion detection. ACM Trans Intell Syst Technol (TIST) 10(3):1–27

    Article  Google Scholar 

  11. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst 27

  12. Lee J, Park K (2021) GAN-based imbalanced data intrusion detection system. Pers Ubiquit Comput 25(1):121–128

    Article  Google Scholar 

  13. Lin Z, Shi Y, Xue Z (2022) IDSGAN: Generative adversarial networks for attack generation against intrusion detection. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 79–91

  14. Ding H, Chen L, Dong L, Fu Z, Cui X (2022) Imbalanced data classification: A KNN and generative adversarial networks-based hybrid approach for intrusion detection. Futur Gener Comput Syst 131:240–254

    Article  Google Scholar 

  15. He X, Chen Q, Tang L, Wang W, Liu T (2022) Cgan-based collaborative intrusion detection for uav networks: A blockchain-empowered distributed federated learning approach. IEEE Internet Things J 10(1):120–132

    Article  Google Scholar 

  16. Hochreiter S, Schmidhuber J (1997) Long Short-Term Memory. Neural Comput 9(8):1735–1780

    Article  CAS  PubMed  Google Scholar 

  17. Lin SZ, Shi Y, Xue Z (2018) Character-level intrusion detection based on convolutional neural networks. In International Joint Conference on Neural Networks (IJCNN), pages 1–8

  18. Aydın H, Orman Z, Aydın MA (2022) A long short-term memory (LSTM)-based distributed denial of service (DDoS) detection and defense system design in public cloud network environment. Comput Secur 118:102725

    Article  Google Scholar 

  19. Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv preprint arXiv:1508.01991

  20. Roy B, Cheung H (2018) A deep learning approach for intrusion detection in internet of things using bi-directional long short-term memory recurrent neural network. In International Telecommunication Networks and Applications Conference (ITNAC), pages 1–6

  21. Kim J, Kim J, Thu HLT, Kim H (2016) Long short term memory recurrent neural network classifier for intrusion detection. In International Conference on Platform Technology and Service (PlatCon), pages 1–5

  22. Althubiti SA, Jones EM, Roy K (2018) LSTM for Anomaly-Based Network Intrusion Detection. In International Telecommunication Networks and Applications Conference (ITNAC), pages 1–3

  23. Imrana Y, Xiang Y, Ali L, Abdul-Rauf Z (2021) A bidirectional LSTM deep learning approach for intrusion detection. Expert Syst Appl 185:115524

    Article  Google Scholar 

  24. Shitharth S, Satheesh N, Kumar BP, Sangeetha K (2021) IDS detection based on optimization based on WI-CS and GNN algorithm in SCADA network. Architectural Wireless Networks Solutions and Security Issues 247–265

  25. Ling C, Zhao X, Lu J, Deng C, Zheng C, Wang J, Chowdhury T, Li Y, Cui H, Zhao T et al (2023) Beyond one-model-fits-all: A survey of domain specialization for large language models. arXiv preprint arXiv:2305.18703

  26. Devlin J, Chang MW, Lee K, Toutanova K (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805

  27. Yin J, Tang MJ, Cao Jinli, Wang Hua (2020) Apply transfer learning to cybersecurity: Predicting exploitability of vulnerabilities by description. Knowl-Based Syst 210:106529

    Article  Google Scholar 

  28. Lee Y, Kim J, Kang P (2021) LAnoBERT: System log anomaly detection based on bert masked language model. arXiv preprint arXiv:2111.09564

  29. Alkhatib N, Mushtaq M, Ghauch H, Danger JL (2022) CAN-BERT do it? controller area network intrusion detection system based on bert language model. In IEEE/ACS 19th International Conference on Computer Systems and Applications (AICCSA), pages 1–8

  30. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  31. Douzas G, Bacao F (2018) Effective data generation for imbalanced learning using conditional generative adversarial networks. Expert Syst Appl 91:464–471

    Article  Google Scholar 

  32. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30

  33. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  34. Salem M, Taheri S, Yuan JS (2018) Anomaly Generation Using Generative Adversarial Networks in Host-Based Intrusion Detection. In IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pages 683–687

Download references

Acknowledgements

The authors gratefully acknowledge the financial assistance provided by the National Natural Science Foundation of China, the Natural Science Foundation of Jiangsu Province, and other research projects.

Funding

This work was supported in part by National Key R &D Program of China under Grant 2021YFB2012301, the National Natural Science Foundation of China under Grants 61502230, 61501224, and 62202221, the Natural Science Foundation of Jiangsu Province under Grant BK20201357 and BK20220331, the Six Talent Peaks Project in Jiangsu Province under Grant RJFW-020.

Author information

Authors and Affiliations

  1. College of Computer and Information Engineering (College of Artificial Intelligence), Nanjing Tech University, Nanjing, 211816, China

    Fang Li, Hang Shen, Jieai Mai, Tianjing Wang & Yuanfei Dai

  2. School of Mechanical and Power Engineering, Nanjing Tech University, Nanjing, 211816, China

    Xiaodong Miao

Authors
  1. Fang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hang Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jieai Mai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianjing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuanfei Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaodong Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Fang Li, Hang Shen, and Jieai Mai wrote the main manuscript text. Tianjing Wang, Yuanfei Dai, and Xiaodong Miao provided guiding ideas and suggestions. All authors reviewed the manuscript.

Corresponding author

Correspondence to Hang Shen.

Ethics declarations

Ethical approval and consent to participate

This article contains no studies with human participants or animals performed by any of the authors.

Consent for publication

All authors agree to publish the paper and related research results.

Conflict of interest

We declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. We declare that there is no financial interest/personal relationship which may be considered as potential competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection: Special Issue on 2 - Track on Security and Privacy

Guess Editor: Rongxing Lu

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, F., Shen, H., Mai, J. et al. Pre-trained language model-enhanced conditional generative adversarial networks for intrusion detection. Peer-to-Peer Netw. Appl. 17, 227–245 (2024). https://doi.org/10.1007/s12083-023-01595-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-023-01595-6

Keywords

Search

Navigation