Abstract
Neural network architectures have been used to address multiple real-world problems with high success. Their extension to graph-structured data started recently to be explored. Graph-neural network (GNN) achieved state-of-the-art performance in multiple problems. In highly imbalanced application domains, such as network intrusion problems, GNN was used to model the network topology. However, in this scenario, the class imbalance problem still affects the performance. Another graph-based solution, the graph attention network (GAT) has also been applied to multiple predictive tasks. Although being a promising avenue, graph-based solutions are still under-explored in imbalanced scenarios. This paper proposes two novel graph-based algorithms, the E-ResSAGE and E-ResGAT algorithms, which build on top of the established GraphSAGE and GAT algorithms, respectively. The key idea is to integrate residual learning into the GNN leveraging the available graph information. Residual connections are added as a strategy to deal with the high class imbalance, aiming at retaining the original information and improving the minority classes’ performance. A case study on intrusion detection is provided. Extensive experiments on four recent intrusion detection datasets show the excellent performance of our proposed approaches, especially when predicting minority classes. We demonstrate that embedding residuals in graph-based algorithms presents a strong advantage when learning under imbalanced domains.
Graphical abstract








Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
The link to each dataset is listed in Section 4.1. The code for pre-processing steps is included in the Github repository https://github.com/George730/E-ResGAT.
Code availability
The code for the two proposed models is also included in the Github repository https://github.com/George730/E-ResGAT.
Notes
By replacing addition with concatenation in residual learning, we will learn two different weight matrices for original and residual features, instead of one. This change allows for more model complexity and may deal with more diverse datasets.
References
Alsaedi A, Moustafa N, Tari Z et al (2020) Ton_iot telemetry dataset: A new generation dataset of iot and iiot for data-driven intrusion detection systems. IEEE Access 8:165130–165150
Altaf T, Wang X, Ni W et al (2023) NE-GConv: A lightweight node edge graph convolutional network for intrusion detection. Comput Secur 130(103):285
Cheng Q, Wu C, Zhou S (2021) Discovering attack scenarios via intrusion alert correlation using graph convolutional networks. IEEE Commun Lett 25(5):1564–1567
Dai H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. In: International conference on machine learning. PMLR, pp 2702–2711
Duan G, Lv H, Wang H et al (2022) Application of a dynamic line graph neural network for intrusion detection with semisupervised learning. IEEE Trans Inf Forensics Secur 18:699–714
Habibi Lashkari A, Kaur G, Rahali A (2020) Didarknet: A contemporary approach to detect and characterize the darknet traffic using deep image learning. In: 2020 the 10th international conference on communication and network security, pp 1–13
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Advances in neural information processing systems 30
He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Johnson JM, Khoshgoftaar TM (2020) The effects of data sampling with deep learning and highly imbalanced big data. Inf Syst Front 22(5):1113–1131
Lee J, Park K (2021) Gan-based imbalanced data intrusion detection system. Pers Ubiquit Comput 25(1):121–128
Lehot PG (1974) An optimal algorithm to detect a line graph and output its root graph. J ACM (JACM) 21(4):569–575
Liu X, Ding J, Jin W et al (2021) Graph neural networks with adaptive residual. Adv Neural Inf Process Syst 34:9720–9733
Maas AL, Hannun AY, Ng AY, et al (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml, Citeseer, p 3
Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11)
Moustafa N, Slay J (2015) Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In: 2015 military communications and information systems conference (MilCIS). IEEE, pp 1–6
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml
Nguyen H, Kashef R (2023) TS-IDS: Traffic-aware self-supervised learning for IoT Network Intrusion Detection. Knowl-Based Syst 279(110):966
Paszke A, Gross S, Massa F et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8026–8037
Scarselli F, Gori M, Tsoi AC et al (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80
Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116
Shuaiyi L, Wang K, Zhang L et al (2022) Global-local integration for GNN-based anomalous device state detection in industrial control systems. Expert Syst Appl 209(118):345
Tang J, Gao X, Hu W (2021) Rgln: Robust residual graph learning networks via similarity-preserving mapping on graphs. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2940–2944
Clevert D-A, Unterthiner T, Hochreiter S (2016) Fast and accurate deep network Learning by Exponential Linear Units (ELUs). In: Bengio Y, LeCun Y (eds) 4th International conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings
Gao Mengnan, Wu Lifa, Li Qi, Chen Wei (2023) Anomaly traffic detection in IoT security using graph neural networks. J Inf Secur Appl 76(103):532
Jiang Weiwei (2022) Graph-based Deep Learning for Communication Networks: A Survey. Comput Commun 185:40–54
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the 3rd international conference for learning representations (iclr’15). San Diego 500
Lo WW, Layeghy S, Sarhan M, Gallagher M, Portmann M (2022) E-GraphSAGE: a graph neural network based intrusion detection system for IoT. NOMS 2022-2022 IEEE/IFIP network operations and management symposium, pp 1–9
Veličković P, Cucurull G, Casannova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. Stat 1050(20):10–48550
Wan Y, Liu Y, Wang D et al (2021) Glad-paw: Graph-based log anomaly detection by position aware wighted graph attention network. In: PAKDD (1). Springer, pp 66–77
Wang S, Liu W, Wu J et al (2016) Training deep neural networks on imbalanced data sets. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 4368–4374
Xiao Q, Liu J, Wng Q et al (2020) Towards network anomaly detection using graph embedding. In: International conference on computational science. Springer, pp 156–169
Yang C, Zhou Z, Wen H et al (2020) Mstnn: A graph learning based method for origin-destination traffic prediction. In: ICC 2020-2020 IEEE interntional conference on communications (ICC). IEEE, pp 1–6
Yu F, Koltun V, Funkhouser T (2017) Dilated residual networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 472–480
Zhou J, Xu Z, Rush AM et al (2020) Automating botnet detection with graph neural network. Preprint arXiv:2003.06344
Funding
This project received research support from Mitacs Global Research Internship program (amount: CAD $2,856), from the Natural Sciences and Engineering Research Council of Canada.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Liyan Chang. Liyan Chang and Paula Branco co-wrote and revised the draft of the manuscript and both authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest/Competing interests
The authors do not have any conflict or competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chang, L., Branco, P. Embedding residuals in graph-based solutions: the E-ResSAGE and E-ResGAT algorithms. A case study in intrusion detection. Appl Intell 54, 6025–6040 (2024). https://doi.org/10.1007/s10489-024-05404-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05404-2