short-paper

A Pilot Study on Secure Code Generation with ChatGPT for Web Applications

Authors:

Mahesh Jamdade,

Yi LiuAuthors Info & Claims

ACMSE '24: Proceedings of the 2024 ACM Southeast Conference

Pages 229 - 234

https://doi.org/10.1145/3603287.3651194

Published: 27 April 2024 Publication History

Abstract

Conversational Large Language Models (LLMs), such as ChatGPT, have demonstrated their potent capabilities in natural language processing tasks. This paper presents a pilot study that uses ChatGPT for generating web application code with a specific emphasis on mitigating four prevalent web application vulnerability types: SQL Injection, Cross Site Scripting, Carriage Return Line Feed Injection, and Exposure of Sensitive Information. The paper uses a case study to illustrate how the vulnerabilities in the code are mitigated with the prompts and the subsequent refinements. The study's findings summarize the security concerns in the code generated by ChatGPT, and the paper proposes a prompt pattern designed to help mitigating the potential vulnerabilities.

References

[1]

Jianjun Chen, Jian Jiang, Haixin Duan, Tao Wan, Shuo Chen, Vern Paxson, and Min Yang. 2018. We still Don't Have Secure Cross-Domain Requests: an Empirical Study of CORS. In 27th USENIX Security Symposium (USENIX Security 18). Baltimore, USA, 1079--1093.

[2]

Edgescan. 2023. 2023 Vulnerability Statistics Report. https://www.edgescan.com/intel-hub/stats-report

[3]

Onyeka Ezenwoye, Yi Liu, and Willam Patten. 2020. Classifying Common Security Vulnerabilities by Software Type. In Proceedings of the 32nd International Conference on Software Engineering and Knowledge Engineering (SEKE 2020). Pittsburgh, USA, 61--64.

[4]

Gertjan Franken, Tom Van Goethem, Lieven Desmet, and Wouter Joosen. 2023. A Bug's Life: Analyzing the Lifecycle and Mitigation Process of Content Security Policy Bugs. In 32nd USENIX Security Symposium (USENIX Security 23). Anaheim, USA, 3673--3690.

[5]

Kristi Hines. 2023. History Of ChatGPT: A Timeline Of The Meteoric Rise Of Generative AI Chatbots. https://www.searchenginejournal.com/history-of-chatgpt-timeline/488370/

[6]

Etienne Janot and Pavol Zavarsky. 2008. Preventing SQL Injections in Online Applications: Study, Recommendations and Java Solution Prototype Based on the SQL DOM. In OWASP Application Security Conference.

[7]

Muhammad Fawad Akbar Khan, Max Ramsdell, Erik Falor, and Hamid Karimi. 2023. Assessing the Promise and Pitfalls of ChatGPT for Automated Code Generation. arXiv preprint (2023). https://doi.org/arXiv:2311.02640

[8]

Soheil Khodayari and Giancarlo Pellegrino. 2022. The State of the SameSite: Studying the Usage, Effectiveness, and Adequacy of SameSite Cookies. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, San Francisco, USA.

[9]

Raphaël Khoury, Anderson R Avila, Jacob Brunelle, and Baba Mamadou Camara. 2023. HowSecure is Code Generated by ChatGPT? arXiv preprint arXiv:2304.09655 (2023).

[10]

Miao Liu, Boyu Zhang, Wenbin Chen, and Xunlai Zhang. 2019. A Survey of Exploitation and Detection Methods of XSS Vulnerabilities. IEEE Access 7 (2019). https://doi.org/10.1109/ACCESS.2019.2960449

[11]

MITRE. 2006--2023. CWE-200: Exposure of Sensitive Information to an Unauthorized Actor. https://cwe.mitre.org/data/definitions/200.html

[12]

MITRE. 2006--2023. CWE-79: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting'). https://cwe.mitre.org/data/definitions/79.html

[13]

MITRE. 2006--2023. CWE-89: Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection'). https://cwe.mitre.org/data/definitions/89.html

[14]

MITRE. 2006--2023. CWE-93: Improper Neutralization of CRLF Sequences ('CRLF Injection'). https://cwe.mitre.org/data/definitions/93.html

[15]

Madhav Nair, Rajat Sadhukhan, and Debdeep Mukhopadhyay. 2023. Generating Secure Hardware Using ChatGPT Resistant to CWEs. Cryptology ePrint Archive (2023). https://eprint.iacr.org/2023/212.pdf

[16]

OWASP. 2017. OWASP Top Ten 2017: OWASP A3:2017-Sensitive Data Exposure. https://owasp.org/www-project-top-ten/2017/A3_2017-Sensitive_Data_Exposure

[17]

OWASP. 2021. OWASP Top 10 A01:2021 - Broken Access Control. https://owasp.org/Top10/A01_2021-Broken_Access_Control/

[18]

OWASP. 2021. OWASP Top 10 A03:2021 - Injection. https://owasp.org/Top10/A03_2021-Injection/

[19]

OWASP. 2021. OWASP Top Ten Web Application Security Risks. https://owasp.org/www-project-top-ten/

[20]

Veracode. 2023. State of Software Security 2023: Annual Report on the State of Application Security. https://www.veracode.com/state-of-software-security-report

[21]

Wallam. 2022. CRLF Injection Attack: Examples and Prevention. https://www.wallarm.com/what/crlf-injection-attack

[22]

Kei Wei, Muthusrinivasan Muthuprasanna, and Suraj Kothari. 2006. Preventing SQL Injection Attacks in Stored Procedures. In Australian Software Engineering Conference (ASWEC'06). IEEE, Sydney, Australia.

[23]

Jules White, Sam Hays, Quchen Fu, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. Chatgpt Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design. arXiv preprint arXiv:2303.07839 (2023).

Cited By

Król K(2025)Between Truth and Hallucinations: Evaluation of the Performance of Large Language Model-Based AI Plugins in Website Quality AnalysisApplied Sciences10.3390/app1505229215:5(2292)Online publication date: 20-Feb-2025
https://doi.org/10.3390/app15052292
Smutny PBojko M(2024)Comparative Analysis of Chatbots Using Large Language Models for Web Development TasksApplied Sciences10.3390/app14211004814:21(10048)Online publication date: 4-Nov-2024
https://doi.org/10.3390/app142110048
Xu HOmitaomu FSabri SZlatanova SLi XSong Y(2024)Leveraging generative AI for urban digital twins: a scoping review on the autonomous generation of urban data, scenarios, designs, and 3D city models for smart city advancementUrban Informatics10.1007/s44212-024-00060-w3:1Online publication date: 14-Oct-2024
https://doi.org/10.1007/s44212-024-00060-w

Index Terms

A Pilot Study on Secure Code Generation with ChatGPT for Web Applications
1. Security and privacy
  1. Software and application security
    1. Web application security

Recommendations

Detection, Avoidance, and Attack Pattern Mechanisms in Modern Web Application Vulnerabilities: Present and Future Challenges

In this paper, we present comprehensive survey of secured web application by identifying numerous serious threats faced by several-related organizations. Based on this, we have summarized the statistics of all emerging web application vulnerabilities by ...
Exploring Defense of SQL Injection Attack in Penetration Testing

SQLIA is adopted to attack websites with and without confidential information. Hackers utilized the compromised website as intermediate proxy to attack others for avoiding being committed of cyber-criminal and also enlarging the scale of Distributed ...
Developing secure web applications

The security of web applications is an important issue for any organisation that deploys its own websites. If an organisation takes the required precautions and countermeasures, it can prevent the possible attacks. Otherwise, its critical data, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ACMSE '24: Proceedings of the 2024 ACM Southeast Conference

April 2024

337 pages

ISBN:9798400702372

DOI:10.1145/3603287

Organizing Chair:
Dan Lo,
Program Chair:
Eric Gamess

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

ACM SE '24

Sponsor:

ACM

ACM SE '24: 2024 ACM Southeast Conference

April 18 - 20, 2024

GA, Marietta, USA

Acceptance Rates

ACMSE '24 Paper Acceptance Rate 44 of 137 submissions, 32%;

Overall Acceptance Rate 502 of 1,023 submissions, 49%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
165
Total Downloads

Downloads (Last 12 months)165
Downloads (Last 6 weeks)16

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Król K(2025)Between Truth and Hallucinations: Evaluation of the Performance of Large Language Model-Based AI Plugins in Website Quality AnalysisApplied Sciences10.3390/app1505229215:5(2292)Online publication date: 20-Feb-2025
https://doi.org/10.3390/app15052292
Smutny PBojko M(2024)Comparative Analysis of Chatbots Using Large Language Models for Web Development TasksApplied Sciences10.3390/app14211004814:21(10048)Online publication date: 4-Nov-2024
https://doi.org/10.3390/app142110048
Xu HOmitaomu FSabri SZlatanova SLi XSong Y(2024)Leveraging generative AI for urban digital twins: a scoping review on the autonomous generation of urban data, scenarios, designs, and 3D city models for smart city advancementUrban Informatics10.1007/s44212-024-00060-w3:1Online publication date: 14-Oct-2024
https://doi.org/10.1007/s44212-024-00060-w

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten