Skip to main content
SpringerLink
Log in

Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT

  • Conference paper
  • First Online:
Computer Security. ESORICS 2023 International Workshops (ESORICS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14399))

Included in the following conference series:

  • 184 Accesses

Abstract

A typical analyst spends much time and effort investigating alerts from network intrusion detection systems (NIDS). Available NIDS rules for enterprise and industrial control systems are not always accompanied by high-level explanations that allow for building valid hypotheses about the attacker’s techniques and intentions. The plethora of rules and the lack of high-level information necessitates new automated methods for alert enrichment. Large language models, such as ChatGPT, encompass a vast amount of knowledge, including cyber threat intelligence such as ports and protocols (low-level) and MITRE ATT &CK techniques (high-level). Despite being a very new technology, ChatGPT is increasingly used in order to automate processes that experts previously performed. In this paper, we explore the ability of ChatGPT to reason about NIDS rules while labeling them with MITRE ATT &CK techniques. We discuss prompt design and present results on ChatGPT-3.5, ChatGPT-4, and a keyword-based approach. Our results indicate that both versions of ChatGPT outperform a baseline that relies on a-priori frequencies of the techniques. ChatGPT-3.5 is much more precise than ChatGPT-4, with a little reduction in recall.

Supported by the U.S.-Israel Energy Center managed by the Israel-U.S. Binational Industrial Research and Development (BIRD) Foundation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://attack.mitre.org/.

  2. 2.

    https://www.snort.org/.

  3. 3.

    https://openai.com/gpt-4.

  4. 4.

    https://zeek.org/.

  5. 5.

    https://github.com/NirDaniel/Labeling-NIDS-Rules-with-MITRE-ATT-CK-Techniques-using-ChatGPT.git.

References

  1. Arafune, M., et al.: Design and development of automated threat hunting in industrial control systems. In: 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), pp. 618–623. IEEE (2022)

    Google Scholar 

  2. Bagui, S.S., et al.: Introducing UWF-ZeekData22: a comprehensive network traffic dataset based on the MITRE ATT &CK framework. Data 8(1), 18 (2023)

    Article  Google Scholar 

  3. Chakrabarti, S., Chakraborty, M., Mukhopadhyay, I.: Study of snort-based IDS. In: Proceedings of the International Conference and Workshop on Emerging Trends in Technology, pp. 43–47 (2010)

    Google Scholar 

  4. Chismon, D., Ruks, M.: Threat intelligence: collecting, analysing, evaluating. MWR InfoSecurity Ltd. 3(2), 36–42 (2015)

    Google Scholar 

  5. Daszczyszak, R., Ellis, D., Luke, S., Whitley, S.: Ttp-based Hunting. Tech. rep, MITRE CORP MCLEAN VA (2019)

    Google Scholar 

  6. Elitzur, A., Puzis, R., Zilberman, P.: Attack hypothesis generation. In: 2019 European Intelligence and Security Informatics Conference (EISIC), pp. 40–47. IEEE (2019)

    Google Scholar 

  7. Gjerstad, J.L.: Generating labelled network datasets of APT with the MITRE CALDERA framework, Master’s thesis (2022)

    Google Scholar 

  8. Haddad, A., Aaraj, N., Nakov, P., Mare, S.F.: Automated mapping of CVE vulnerability records to MITRE CWE weaknesses. arXiv preprint arXiv:2304.11130 (2023)

  9. Husari, G., Al-Shaer, E., Ahmed, M., Chu, B., Niu, X.: Ttpdrill: automatic and accurate extraction of threat actions from unstructured text of CTI sources. In: Proceedings of the 33rd Annual Computer Security Applications Conference, pp. 103–115 (2017)

    Google Scholar 

  10. Kaiser, F.K., et al.: Attack hypotheses generation based on threat intelligence knowledge graph. IEEE Trans. Dependable Secure Comput. 20, 4793–4809 (2023)

    Article  Google Scholar 

  11. Khamphakdee, N., Benjamas, N., Saiyod, S.: Improving intrusion detection system based on Snort rules for network probe attack detection. In: 2014 2nd International Conference on Information and Communication Technology (ICoICT), pp. 69–74. IEEE (2014)

    Google Scholar 

  12. Legoy, V., Caselli, M., Seifert, C., Peter, A.: Automated retrieval of ATT &CK tactics and techniques for cyber threat reports. arXiv preprint arXiv:2004.14322 (2020)

  13. Li, Z., Zeng, J., Chen, Y., Liang, Z.: Attackg: Constructing technique knowledge graph from cyber threat intelligence reports. In: Atluri, V., Di Pietro, R., Jensen, C.D., Meng, W. (eds.) Computer Security – ESORICS 2022. ESORICS 2022. Lecture Notes in Computer Science, vol. 13554, pp. 589–609. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-17140-6_29

  14. Liao, X., Yuan, K., Wang, X., Li, Z., Xing, L., Beyah, R.: Acing the IOC game: toward automatic discovery and analysis of open-source cyber threat intelligence. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 755–766 (2016)

    Google Scholar 

  15. Lin, S.X., Li, Z.J., Chen, T.Y., Wu, D.J.: Attack tactic labeling for cyber threat hunting. In: 2022 24th International Conference on Advanced Communication Technology (ICACT), pp. 34–39. IEEE (2022)

    Google Scholar 

  16. Long, C., et al.: Evaluating ChatGPT4 in Canadian otolaryngology-head and neck surgery board examination using the CVSA model. medRxiv pp. 2023–05 (2023)

    Google Scholar 

  17. McPhee, M.: Methods to employ zeek in detecting MITRE ATT &CK techniques, Tech. Rep. (2020)

    Google Scholar 

  18. Mendsaikhan, O., Hasegawa, H., Yamaguchi, Y., Shimada, H.: Automatic mapping of vulnerability information to adversary techniques. In: The Fourteenth International Conference on Emerging Security Information, Systems and Technologies SECUREWARE2020 (2020)

    Google Scholar 

  19. Palacin, V.: Practical Threat Intelligence and Data-driven Threat Hunting. Packt Publishing (2021)

    Google Scholar 

  20. Peng, Y., Wang, H.: Design and implementation of network instruction detection system based on snort and NTOP. In: 2012 International Conference on Systems and Informatics (ICSAI2012), pp. 116–120. IEEE (2012)

    Google Scholar 

  21. Rani, N., Saha, B., Maurya, V., Shukla, S.K.: TTPHunter: automated extraction of actionable intelligence as TTPs from narrative threat reports. In: Proceedings of the 2023 Australasian Computer Science Week, pp. 126–134 (2023)

    Google Scholar 

  22. Satvat, K., Gjomemo, R., Venkatakrishnan, V.: Extractor: extracting attack behavior from threat reports. In: 2021 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 598–615. IEEE (2021)

    Google Scholar 

  23. Sentonas, M.: Crowdstrike introduces Charlotte AI, generative AI security analyst - crowdstrike (2023). https://www.crowdstrike.com/blog/crowdstrike-introduces-charlotte-ai-to-deliver-generative-ai-powered-cybersecurity/

  24. Shackleford, D.: Who’s using cyberthreat intelligence and how. SANS Institute (2015)

    Google Scholar 

  25. Sharma, Y., Birnbach, S., Martinovic, I.: Radar: Effective network-based malware detection based on the MITRE ATT &CK framework. arXiv preprint arXiv:2212.03793 (2022)

  26. Strom, B.E., Applebaum, A., Miller, D.P., Nickels, K.C., Pennington, A.G., Thomas, C.B.: MITRE ATT &CK®: Design and philosophy (2020)

    Google Scholar 

  27. Tod-Răileanu, G., Axinte, S.D.: ChatGPT-information security overview. In: International Conference on Cybersecurity and Cybercrime, vol. 10 (2023)

    Google Scholar 

  28. Törnberg, P.: Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv preprint arXiv:2304.06588 (2023)

  29. Vulnerabilities, C.: Common vulnerabilities and exposures (2005). https://www.cve.org/About/Metrics

  30. You, Y., et al.: TIM: threat context-enhanced TTP intelligence mining on unstructured threat data. Cybersecurity 5(1), 3 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Software and Information Systems Engineering, Ben-Gurion University, Beer-Sheva, Israel

    Nir Daniel, Anton Dzega & Rami Puzis

  2. Cyber@BGU, Cyber Labs at Ben-Gurion University, Beer-Sheva, Israel

    Nir Daniel, Anton Dzega, Aviad Elyashar & Rami Puzis

  3. Institute for Industrial Production, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Florian Klaus Kaiser

  4. Institute of Information Security and Dependability, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Florian Klaus Kaiser

  5. Department of Computer Science, Shamoon College of Engineering, Beer-Sheva, Israel

    Aviad Elyashar

Authors
  1. Nir Daniel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Florian Klaus Kaiser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anton Dzega
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aviad Elyashar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rami Puzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nir Daniel .

Editor information

Editors and Affiliations

  1. Norwegian University of Science and Technology, Gjøvik, Norway

    Sokratis Katsikas

  2. Norwegian Computing Center, Oslo, Norway

    Habtamu Abie

  3. University of Trento, Trento, Italy

    Silvio Ranise

  4. University of Genoa, Genoa, Italy

    Luca Verderame

  5. Consiglio Nazionale delle Ricerche (CNR), Genoa, Italy

    Enrico Cambiaso

  6. SINTEF A.S., Oslo, Norway

    Rita Ugarelli

  7. Instituto Superior de Engenharia do Porto, Porto, Portugal

    Isabel Praça

  8. Hong Kong Polytechnic University, Hong Kong, China

    Wenjuan Li

  9. Technical University of Denmark, Kongens Lyngby, Denmark

    Weizhi Meng

  10. University of Nottingham, Nottingham, UK

    Steven Furnell

  11. Norwegian University of Science and Technology, Gjøvik, Norway

    Basel Katt

  12. Norwegian Computing Center, Oslo, Norway

    Sandeep Pirbhulal

  13. Institute for Energy Technology (IFE), Halden, Norway

    Ankur Shukla

  14. University of Calabria, Rende, Italy

    Michele Ianni

  15. University of Verona, Verona, Italy

    Mila Dalla Preda

  16. The University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

  17. University of Lisbon, Lisbon, Portugal

    Miguel Pupo Correia

  18. University of Twente, Enschede, The Netherlands

    Abhishta Abhishta

  19. University of Amsterdam, Amsterdam, The Netherlands

    Giovanni Sileno

  20. Open University in the Netherlands, Heerlen, The Netherlands

    Mina Alishahi

  21. Robert Gordon University, Aberdeen, UK

    Harsha Kalutarage

  22. Osaka University, Osaka, Japan

    Naoto Yanai

A Appendix: Chat-GPT Prompt Templates

A Appendix: Chat-GPT Prompt Templates

  • ChatGPT-Prompt for the WLT method “You are going to receive a Snort rule and your task is to find as many MITRE ATT &CK techniques as possible that are associated with the rule. Note: You should categorize the techniques to 1 or 2. Technique of type 1 is a technique that you can associate with the rule directly based on the rule. Technique of type 2 is a technique that can be associated with the rule indirectly, based on your knowledge and understanding. The categorization value should be the value 1 or 2, based on the explanation given above. The quotes field value should contain quotes from the rules data that are relevant to the technique mapped and they are the main reason you believe the mapping to this technique is correct. The explanation’s value should be your explanation for why you decided to give the technique and how it is associated with the rule. The technique id should be the official MITRE technique id. For each technique include the following information as JSON: sid, Technique id, Technique name, Categorization, Quotes, Explanation. After each rule I will provide you with, answer according to the provided format. Please do not write anything else but the JSON. Rule: {Snort rule}”

  • ChatGPT-Prompt for the LT method “I will provide you with some knowledge now on MITRE ATT &CK techniques, then you are going to receive a task, you may use the knowledge below to perform the task: {List of MITRE ATT &CK techniques IDs and names}. You are going to receive a Snort rule and your task is to find as many MITRE ATT &CK techniques as possible that are associated with the rule. Note: You should categorize the techniques to 1 or 2. Technique of type 1 is a technique that you can associate with the rule directly based on the rule. Technique of type 2 is a technique that can be associated with the rule indirectly, based on your knowledge and understanding. The categorization value should be the value 1 or 2, based on the explanation given above. The quotes field value should contain quotes from the rules data that are relevant to the technique mapped and they are the main reason you believe the mapping to this technique is correct. The explanation’s value should be your explanation for why you decided to give the technique and how it is associated with the rule. The technique id should be the official MITRE technique id. For each technique include the following information as JSON: sid, Technique id, Technique name, Categorization, Quotes, Explanation. After each rule I will provide you with, answer according to the provided format. Please do not write anything else but the JSON. Rule: {Snort rule}”

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Daniel, N., Kaiser, F.K., Dzega, A., Elyashar, A., Puzis, R. (2024). Labeling NIDS Rules with MITRE ATT &CK Techniques Using ChatGPT. In: Katsikas, S., et al. Computer Security. ESORICS 2023 International Workshops. ESORICS 2023. Lecture Notes in Computer Science, vol 14399. Springer, Cham. https://doi.org/10.1007/978-3-031-54129-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-54129-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-54128-5

  • Online ISBN: 978-3-031-54129-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation