skip to main content
10.1145/3689217.3690616acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article
Open access

Towards Large Language Model (LLM) Forensics Using LLM-based Invocation Log Analysis

Published: 19 November 2024 Publication History

Abstract

Large Language Models (LLMs) have fostered the emergence of software application architectures that improve user experiences powered by generative artificial intelligence. A range of cyber attacks are possible against an LLM. A novel approach to digital forensic analysis of LLM-integrated applications is presented for prompt injection attacks. The forensic analysis process is invoked through LLM log analysis. We propose LLM invocation logging as a critical component for enhancing digital forensic readiness in LLM-integrated applications and evaluate 13 state-of-the-art LLMs for this analysis task. Our findings demonstrate the potential utility of selected LLMs in the context of prompt-to-SQL attacks, influenced by sampling temperature and context window size parameters. We also identify limitations of our work and propose key areas for future research, for ongoing contribution to the emerging field of LLM forensics.

References

[1]
S. Abdelnabi, K. Greshake, S. Mishra, C. Endres, T. Holz, and M. Fritz. 2023. Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In AISec 2023 - Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. 79--90. https://doi.org/10.1145/3605764.3623985
[2]
Kenny Awuson-David, Tawfik Al-Hadhrami, Mamoun Alazab, Nazaraf Shah, and Andrii Shalaginov. 2021. BCFL Logging: An Approach to Acquire and Preserve Admissible Digital Forensics Evidence in Cloud Ecosystem. Future Generation Computer Systems, Vol. 122 (Sept. 2021), 1--13. https://doi.org/10.1016/j.future.2021.03.001
[3]
Harrison Chase. 2022. LangChain. https://github.com/langchain-ai/langchain
[4]
Xiaoyu Du, Chris Hargreaves, John Sheppard, Felix Anda, Asanka Sayakkara, Nhien-An Le-Khac, and Mark Scanlon. 2020. SoK: Exploring the State of the Art and the Future Potential of Artificial Intelligence in Digital Forensic Investigation. In Proceedings of the 15th International Conference on Availability, Reliability and Security (ARES '20). Association for Computing Machinery, New York, NY, USA, 1--10. https://doi.org/10.1145/3407023.3407068
[5]
Chris Egersdoerfer, Di Zhang, and Dong Dai. 2023. Early Exploration of Using ChatGPT for Log-based Anomaly Detection on Parallel File Systems Logs. In Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing (HPDC '23). Association for Computing Machinery, New York, NY, USA, 315--316. https://doi.org/10.1145/3588195.3595943
[6]
Mohamed Elyas, Atif Ahmad, Sean B. Maynard, and Andrew Lonie. 2015. Digital Forensic Readiness: Expert Perspectives on a Theoretical Framework. Computers & Security, Vol. 52 (July 2015), 70--89. https://doi.org/10.1016/j.cose.2015.04.003
[7]
A. Esmradi, D.W. Yip, and C.F. Chan. 2024. A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models. Communications in Computer and Information Science, Vol. 2034 CCIS (2024), 76--95. https://doi.org/10.1007/978--981--97--1274--8_6
[8]
Ben Martini and Kim-Kwang Raymond Choo. 2012. An Integrated Conceptual Digital Forensic Framework for Cloud Computing. Digital Investigation, Vol. 9, 2 (Nov. 2012), 71--80. https://doi.org/10.1016/j.diin.2012.07.001
[9]
G. Michelet and F. Breitinger. 2024. ChatGPT, Llama, Can You Write My Report? An Experiment on Assisted Digital Forensics Reports Written Using (Local) Large Language Models. Forensic Science International: Digital Investigation, Vol. 48 (2024). https://doi.org/10.1016/j.fsidi.2023.301683
[10]
Microsoft. 2024. Implement Logging and Monitoring for Azure OpenAI Language Models - Azure Architecture Center. https://learn.microsoft.com/en-us/azure/architecture/ai-ml/openai/architecture/log-monitor-azure-openai.
[11]
MITRE. 2023. Achieving Code Execution in MathGPT via Prompt Injection textbar MITRE ATLAS?. https://atlas.mitre.org/studies/AML.CS0016.
[12]
Rodrigo Pedro, Daniel Castro, Paulo Carreira, and Nuno Santos. 2023. From Prompt Injections to SQL Injection Attacks: How Protected Is Your LLM-Integrated Web Application? https://doi.org/10.48550/arXiv.2308.01990 arxiv: 2308.01990 [cs]
[13]
Sebastien Philomin, Avinash Singh, Adeyemi Ikuesan, and Hein Venter. 2020. Digital Forensic Readiness Framework for Smart Homes. In International Conference on Cyber Warfare and Security. Academic Conferences International Limited, Reading, United Kingdom, 627--636,XVI,XVIII. https://doi.org/10.34190/ICCWS.20.047
[14]
Ameer Pichan, Mihai Lazarescu, and Sie Teng Soh. 2018. Towards a Practical Cloud Forensics Logging Framework. Journal of Information Security and Applications, Vol. 42 (Oct. 2018), 18--28. https://doi.org/10.1016/j.jisa.2018.07.008
[15]
Matthew Renze and Erhan Guven. 2024. The Effect of Sampling Temperature on Problem Solving in Large Language Models. https://doi.org/10.48550/arXiv.2402.05201 arxiv: 2402.05201 [cs]
[16]
Mark Scanlon, Frank Breitinger, Christopher Hargreaves, Jan-Niclas Hilgert, and John Sheppard. 2023. ChatGPT for Digital Forensic Investigation: The Good, the Bad, and the Unknown. Forensic Science International: Digital Investigation, Vol. 46 (Oct. 2023), 301609. https://doi.org/10.1016/j.fsidi.2023.301609
[17]
Mark Scanlon, Bruce Nikkel, and Zeno Geradts. 2023. Digital Forensic Investigation in the Age of ChatGPT. Forensic Science International: Digital Investigation, Vol. 44 (March 2023), 301543. https://doi.org/10.1016/j.fsidi.2023.301543
[18]
Avinash Singh, Adeyemi R. Ikuesan, and Hein S. Venter. 2019. Digital Forensic Readiness Framework for Ransomware Investigation. In Digital Forensics and Cyber Crime, Frank Breitinger and Ibrahim Baggili (Eds.). Springer International Publishing, Cham, 91--105. https://doi.org/10.1007/978--3-030-05487--8_5
[19]
E.K. Sreya, Sakshi, and M. Wadhwa. 2023. Enhancing Digital Investigation: Leveraging ChatGPT for Evidence Identification and Analysis in Digital Forensics. In Proceedings - 4th IEEE 2023 International Conference on Computing, Communication, and Intelligent Systems, ICCCIS 2023. 733--738. https://doi.org/10.1109/ICCCIS60361.2023.10425000
[20]
John Tan. 2001. Forensic readiness. Cambridge, MA:@ Stake, Vol. 1 (2001).
[21]
Aleksandar Valjarevic and H. S. Venter. 2011. Towards a Digital Forensic Readiness Framework for Public Key Infrastructure Systems. In 2011 Information Security for South Africa. IEEE, Johannesburg, South Africa, 1--10. https://doi.org/10.1109/ISSA.2011.6027536
[22]
Aleksandar Valjarevic and Hein S. Venter. 2012. Harmonised Digital Forensic Investigation Process Model. In 2012 Information Security for South Africa. 1--10. https://doi.org/10.1109/ISSA.2012.6320441
[23]
Akila Wickramasekara and Mark Scanlon. 2024. A Framework for Integrated Digital Forensic Investigation Employing AutoGen AI Agents. In 2024 12th International Symposium on Digital Forensics and Security (ISDFS). 01--06. https://doi.org/10.1109/ISDFS60797.2024.10527235
[24]
Shams Zawoad, Amit Kumar Dutta, and Ragib Hasan. 2013. SecLaaS: Secure Logging-as-a-Service for Cloud Forensics. In Proceedings of the 8th ACM SIGSAC Symposium on Information, Computer and Communications Security (ASIA CCS '13). Association for Computing Machinery, New York, NY, USA, 219--230. https://doi.org/10.1145/2484313.2484342
[25]
Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. https://doi.org/10.48550/arXiv.2403.02691 arxiv: 2403.02691 [cs]

Index Terms

  1. Towards Large Language Model (LLM) Forensics Using LLM-based Invocation Log Analysis

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    LAMPS '24: Proceedings of the 1st ACM Workshop on Large AI Systems and Models with Privacy and Safety Analysis
    November 2024
    111 pages
    ISBN:9798400712098
    DOI:10.1145/3689217
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 November 2024

    Check for updates

    Author Tags

    1. digital forensics
    2. large language model (llm)
    3. llm-integrated applications
    4. log analysis

    Qualifiers

    • Research-article

    Conference

    CCS '24
    Sponsor:

    Upcoming Conference

    CCS '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 191
      Total Downloads
    • Downloads (Last 12 months)191
    • Downloads (Last 6 weeks)102
    Reflects downloads up to 10 Feb 2025

    Other Metrics

    Citations

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media