Skip to main content

Advertisement

Springer Nature Link
Log in

Attack stage detection method based on vector reconstruction error autoencoder and explainable artificial intelligence

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

One of the most serious security threats faced by the Internet today is multi-stage attacks. In response to this challenge, anomaly detection-based methods have been widely used to identify different stages of such attacks. However, current anomaly detection approaches for detecting attack stages face several challenges: (1) Traditional methods often adopt a global perspective, lacking detailed consideration of the traffic characteristics at each stage, which may reduce the accuracy in detecting specific stages. (2) Many detection methods rely on deep learning models with complex internal structures, making their decision-making process opaque and difficult for users to interpret. This also complicates model optimization and improvement. To address these challenges, this paper proposes an attack stage detection method based on a vector reconstruction error autoencoder. By analyzing each stage independently, the proposed method enhances detection precision. It also integrates the permutation feature importance technique to quantify and interpret the model’s reliance on different features, guiding feature selection and model optimization. Experiments conducted using the CIC-IDS2017 and CSE-CIC-IDS2018 datasets demonstrate that the proposed method achieves higher accuracy, precision, recall, and F1 score compared to other methods, confirming its feasibility and effectiveness in detecting multi-stage attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

No datasets were generated or analyzed during the current study.

References

  1. Moustafa N, Hu J, Slay J (2019) A holistic review of network anomaly detection systems: a comprehensive survey. J Netw Comput Appl 128:33–55. https://doi.org/10.1016/j.jnca.2018.12.006

    Article  Google Scholar 

  2. Navarro J, Deruyver A, Parrend P (2018) A systematic survey on multi-step attack detection. Comput Secur 76:214–249. https://doi.org/10.1016/j.cose.2018.03.001

    Article  Google Scholar 

  3. Milajerdi SM, Gjomemo R, Eshete B, et al (2019) Holmes: real-time apt detection through correlation of suspicious information flows. In: 2019 IEEE Symposium on Security and Privacy (SP), IEEE, pp 1137–1152. https://doi.org/10.1109/SP.2019.00026

  4. Husák M, Komárková J, Bou-Harb E et al (2018) Survey of attack projection, prediction, and forecasting in cyber security. IEEE Communications Surveys & Tutorials 21(1):640–660. https://doi.org/10.1109/COMST.2018.2871866

    Article  Google Scholar 

  5. Zhang X, Wu T, Zheng Q et al (2022) Multi-step attack detection based on pre-trained hidden Markov models. Sensors 22(8):2874. https://doi.org/10.3390/s22082874

    Article  Google Scholar 

  6. Farhat S, Abdelkader M, Meddeb-Makhlouf A et al (2023) Cads-ml/dl: efficient cloud-based multi-attack detection system. Int J Inf Secur 22(6):1989–2013. https://doi.org/10.1007/s10207-023-00729-4

    Article  Google Scholar 

  7. Panahnejad M, Mirabi M (2022) Apt-dt-kc: advanced persistent threat detection based on kill-chain model. J Supercomput 78(6):8644–8677. https://doi.org/10.1007/s11227-021-04201-9

    Article  Google Scholar 

  8. Kumar A, Noliya A, Makani R (2024) Fuzzy inference based feature selection and optimized deep learning for advanced persistent threat attack detection. Int J Adapt Control Signal Process 38(2):604–620. https://doi.org/10.1002/acs.3717

    Article  Google Scholar 

  9. Al-Saraireh J et al (2022) A novel approach for detecting advanced persistent threats. Egyptian Inf J 23(4):45–55. https://doi.org/10.1016/j.eij.2022.06.005

    Article  Google Scholar 

  10. Ahmed Y, Asyhari AT, Rahman MA (2021) A cyber kill chain approach for detecting advanced persistent threats. Comput Mater Continua 67(2), 2497–2513. https://doi.org/10.32604/cmc.2021.014223

  11. Xie L, Li X, Yang H, et al (2022) A multi-stage apt attack detection method based on sample enhancement. In: International Symposium on Cyberspace Safety and Security, Springer, pp 209–216. https://doi.org/10.1007/978-3-031-18067-5_15

  12. Zhou P, Zhou G, Wu D et al (2021) Detecting multi-stage attacks using sequence-to-sequence model. Comput Secur 105:102203. https://doi.org/10.1016/j.cose.2021.102203

    Article  Google Scholar 

  13. Li T, Liu Y, Liu Y et al (2020) Attack plan recognition using hidden Markov and probabilistic inference. Comput Secur 97:101974. https://doi.org/10.1016/j.cose.2020.101974

    Article  Google Scholar 

  14. Shin J, Choi SH, Liu P et al (2019) Unsupervised multi-stage attack detection framework without details on single-stage attacks. Futur Gener Comput Syst 100:811–825. https://doi.org/10.1016/j.future.2019.05.032

    Article  Google Scholar 

  15. Kumar A, Thing VL (2023) Raptor: advanced persistent threat detection in industrial iot via attack stage correlation. In: 2023 20th Annual International Conference on Privacy, Security and Trust (PST), IEEE, pp 1–12. https://doi.org/10.1109/PST58708.2023.10320192

  16. Keshk M, Koroniotis N, Pham N et al (2023) An explainable deep learning-enabled intrusion detection framework in iot networks. Inf Sci 639:119000. https://doi.org/10.1016/j.ins.2023.119000

    Article  Google Scholar 

  17. Oseni A, Moustafa N, Creech G et al (2022) An explainable deep learning framework for resilient intrusion detection in IoT-enabled transportation networks. IEEE Trans Intell Transp Syst 24(1):1000–1014. https://doi.org/10.1109/TITS.2022.3188671

    Article  Google Scholar 

  18. Patil S, Varadarajan V, Mazhar SM et al (2022) Explainable artificial intelligence for intrusion detection system. Electronics 11(19):3079. https://doi.org/10.3390/electronics11193079

    Article  Google Scholar 

  19. Abou El Houda Z, Brik B, Khoukhi L (2022) “why should i trust your ids?”: an explainable deep learning framework for intrusion detection systems in internet of things networks. IEEE Open J Commun Soc 3:1164–1176. https://doi.org/10.1109/OJCOMS.2022.3188750

  20. Le TTH, Kim H, Kang H et al (2022) Classification and explanation for intrusion detection system based on ensemble trees and shap method. Sensors 22(3):1154. https://doi.org/10.3390/s22031154

    Article  Google Scholar 

  21. Chou D, Jiang M (2021) A survey on data-driven network intrusion detection. ACM Comput Surveys (CSUR) 54(9):1–36. https://doi.org/10.1145/3472753

    Article  Google Scholar 

  22. Liu W, Wang Z, Liu X et al (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26. https://doi.org/10.1016/j.neucom.2016.12.038

    Article  Google Scholar 

  23. Torabi H, Mirtaheri SL, Greco S (2023) Practical autoencoder based anomaly detection by using vector reconstruction error. Cybersecurity 6(1):1. https://doi.org/10.1186/s42400-022-00134-9

    Article  Google Scholar 

  24. Khalane A, Makwana R, Shaikh T et al (2023) Evaluating significant features in context-aware multimodal emotion recognition with XAI methods. Expert Syst. https://doi.org/10.1111/exsy.13403

    Article  Google Scholar 

  25. Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6:52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052

    Article  Google Scholar 

  26. Du M, Liu N, Hu X (2019) Techniques for interpretable machine learning. Commun ACM 63(1):68–77. https://doi.org/10.1145/3359786

    Article  Google Scholar 

  27. Breiman L (2001) Random forests. Machine learning 45:5–32. https://doi.org/10.1023/A:1010933404324

    Article  Google Scholar 

  28. Fisher A, Rudin C, Dominici F (2019) All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. J Mach Learn Res 20(177):1–81. https://doi.org/10.48550/arXiv.1801.01489

  29. Sharafaldin I, Lashkari AH, Ghorbani AA et al (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108–116. https://doi.org/10.5220/0006639801080116

    Article  Google Scholar 

  30. Wang YC, Houng YC, Chen HX et al (2023) Network anomaly intrusion detection based on deep learning approach. Sensors 23(4):2171. https://doi.org/10.3390/s23042171

    Article  Google Scholar 

  31. Gopalan SS, Ravikumar D, Linekar D, et al (2021) Balancing approaches towards ml for ids: a survey for the cse-cic ids dataset. In: 2020 International Conference on Communications, Signal Processing, and Their Applications (ICCSPA), IEEE, pp 1–6. https://doi.org/10.1109/ICCSPA49915.2021.9385742

Download references

Acknowledgements

This work was supported in part by Jilin Province Key Research Development Plan Project (20230203037SF) and Jilin Provincial Science and Technology Department Innovation Platform (Base) and Talent Special Project (20220508043RC).

Author information

Authors and Affiliations

  1. College of Software, Jilin University, Changchun, 130012, China

    Jinze Li & Zichen Qi

  2. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Xiangyu Meng, Dong Guo & Cong Fu

  3. Symbol Computation and Knowledge Engineer of Ministry of Education, Jilin University, Changchun, 130012, China

    Dong Guo

Authors
  1. Jinze Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiangyu Meng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zichen Qi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Dong Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Cong Fu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Conceptualization was contributed by D.G.; methodology was contributed by J.L.,X.M.; software was contributed by X.M.; data curation was contributed by J.L.; validation was contributed by Z.Q.; formal analysis was contributed by Z.Q.; investigation was contributed by J.L.; funding acquisition was contributed by X.M.; resources were contributed by D.G.; writing—original draft preparation was contributed by J.L.; writing—review and editing was contributed by D.G.; visualization was contributed by C.F.; supervision was contributed by D.G.; project administration was contributed by X.M.; all authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Dong Guo.

Ethics declarations

Conflict of interest

The authors declare no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Meng, X., Qi, Z. et al. Attack stage detection method based on vector reconstruction error autoencoder and explainable artificial intelligence. J Supercomput 81, 62 (2025). https://doi.org/10.1007/s11227-024-06473-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06473-3

Keywords