research-article

Open access

Does Generative AI Generate Smells Related to Container Orchestration?: An Exploratory Study with Kubernetes Manifests

Authors:

Rachel Meredith,

Júlia Coriolano,

Muhammad Ali Babar,

Akond RahmanAuthors Info & Claims

MSR '24: Proceedings of the 21st International Conference on Mining Software Repositories

Pages 192 - 196

https://doi.org/10.1145/3643991.3645079

Published: 02 July 2024 Publication History

Abstract

Generative artificial intelligence (AI) technologies, such as ChatGPT have shown promise in solving software engineering problems. However, these technologies have also shown to be susceptible to generating software artifacts that contain quality issues. A systematic characterization of quality issues, such as smells in ChatGPT-generated artifacts can help in providing recommendations for practitioners who use generative AI for container orchestration.

We conduct an empirical study with 98 Kubernetes manifests to quantify smells in manifests generated by ChatGPT. Our empirical study shows: (i) 35.8% of the 98 Kubernetes manifests generated include at least one instance of smell; (ii) two types of objects Kubernetes namely, Deployment and Service are impacted by identified smells; and (iii) the most frequently occurring smell is unset CPU and memory requirements. Based on our findings, we recommend practitioners to apply quality assurance activities for ChatGPT-generated Kubernetes manifests prior to using these manifests for container orchestration.

References

[1]

Anonymous. 2024. K8s-Pod-Example-1. https://chat.openai.com/share/0cee78c4-4290-4bc0-9f2a-a492859d6281. [Online; accessed 20-January-2024].

[2]

Agathe Blaise and Filippo Rebecchi. 2022. Stay at the Helm: secure Kubernetes deployments via graph generation and attack reconstruction. In 2022 IEEE 15th International Conference on Cloud Computing (CLOUD). 59--69.

[3]

Dibyendu Brinto Bose, Akond Rahman, and Shazibul Islam Shamim. 2021. 'Under-reported' Security Defects in Kubernetes Manifests. In 2021 IEEE/ACM 2nd International Workshop on Engineering and Cybersecurity of Critical Systems (EnCyCriS). IEEE, 9--12.

Digital Library

[4]

Canonical. 2021. Kubernetes and cloud native operations report 2021. https://juju.is/cloud-native-kubernetes-usage-report-2021

[5]

Dev Nag. 2023. Overcoming the Kubernetes Skills Gap with ChatGPT Assistance. https://thenewstack.io/overcoming-the-kubernetes-skills-gap-with-chatgpt-assistance/. [Online; accessed 18-November-2023].

[6]

Jayne Groll. 2022. IT talent: 4 ways to address a Kubernetes skills shortage. https://enterprisersproject.com/article/2022/3/address-kubernetes-skills-shortage. [Online; accessed 19-June-2023].

[7]

KubeLinter. 2024. Introduction - KubeLinter. https://docs.kubelinter.io/. [Online; accessed 19-January-2024].

[8]

Kubernetes. 2021. Production-Grade Container Orchestration. https://kubernetes.io/

[9]

Kubernetes. 2023. Case Study: Capital One. https://kubernetes.io/case-studies/capital-one/. [Online; accessed 10-June-2023].

[10]

Janae Lee. 2022. How to beat the Kubernetes skills shortage. https://www.infoworld.com/article/3679749/how-to-beat-the-kubernetes-skills-shortage.html. [Online; accessed 18-June-2023].

[11]

S. Miles. 2020. Kubernetes: A Step-By-Step Guide For Beginners To Build, Manage, Develop, and Intelligently Deploy Applications By Using Kubernetes (2020 Edition). Independently Published. https://books.google.com/books?id=M4VmzQEACAAJ

[12]

Nhan Nguyen and Sarah Nadi. 2022. An empirical evaluation of GitHub copilot's code suggestions. In Proceedings of the 19th International Conference on Mining Software Repositories. 1--5.

Digital Library

[13]

OpenAI. 2022. ChatGPT: Optimizing Language Models for Dialogue. https://openai.com/blog/chatgpt/. [Online; accessed 12-November-2023].

[14]

OpsCruise. 2023. Exploiting ChatGPT for TroubleShooting Kubernetes Problems. https://www.opscruise.com/newsroom-post/exploiting-chatgpt-for-troubleshooting-kubernetes-problems#. [Online; accessed 17-November-2023].

[15]

Akond Rahman. 2023. Verifiability Package for Paper. [Online; accessed 10-December-2023].

[16]

Akond Rahman and Chris Parnin. 2023. Detecting and Characterizing Propagation of Security Weaknesses in Puppet-based infrastructure Management. IEEE Transactions on Software Engineering (2023), 1--18.

Digital Library

[17]

Akond Rahman, Shazibul Islam Shamim, Dibyendu Brinto Bose, and Rahul Pandita. 2023. Security Misconfigurations in Open Source Kubernetes Manifests: An Empirical Study. ACM Trans. Softw. Eng. Methodol. 32, 4, Article 99 (may 2023), 36 pages.

Digital Library

[18]

Renjith Ravindranathan. 2023. ChatGPT for your Kubernetes Cluster --- k8sgpt. https://medium.com/techbeatly/chatgpt-for-your-kubernetes-cluster-k8sgpt-649f2cad1bd5. [Online; accessed 19-November-2023].

[19]

Markets N Research. 2023. Case Study: OpenAI. https://www.globenewswire.com/news-release/2023/03/06/2621358/0/en/Latest-Global-Kubernetes-Market-Size-Share-Worth-USD-7-8-Billion-by-2030-at-an-23-40-CAGR-Markets-N-Research-Share-Trends-Cap-Adoption-Forecast-Segmentation-Growth-Value.html. [Online; accessed 12-June-2023].

[20]

Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2023. Code Llama: Open Foundation Models for Code. (2023).

[21]

Johnny Saldaña. 2015. The coding manual for qualitative researchers. Sage.

[22]

Daniel Schlagwein and Leslie Willcocks. 2023. 'ChatGPT et al.': The ethics of using (generative) artificial intelligence in research and science., 232--238 pages.

[23]

M. Islam Shamim, F. Ahamed Bhuiyan, and A. Rahman. 2020. XI Commandments of Kubernetes Security: A Systematization of Knowledge Related to Kubernetes Security Practices. In 2020 IEEE Secure Development (SecDev). IEEE Computer Society, Los Alamitos, CA, USA, 58--64.

[24]

Tao Xiao, Christoph Treude, Hideaki Hata, and Kenichi Matsumoto. 2024. DevGPT: Studying Developer-ChatGPT Conversations. In Proceedings of the International Conference on Mining Software Repositories (MSR 2024).

Digital Library

Cited By

Faruqui NRaju NSivakumar SPatel NVengaramkode Bhaskaran SKhanam SBhuiyan T(2025)Gen-Optimizer: A Generative AI Framework for Strategic Business Cost OptimizationComputers10.3390/computers1402005914:2(59)Online publication date: 10-Feb-2025
https://doi.org/10.3390/computers14020059

Index Terms

Does Generative AI Generate Smells Related to Container Orchestration?: An Exploratory Study with Kubernetes Manifests
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Security Misconfigurations in Open Source Kubernetes Manifests: An Empirical Study
Context: Kubernetes has emerged as the de-facto tool for automated container orchestration. Business and government organizations are increasingly adopting Kubernetes for automated software deployments. Kubernetes is being used to provision applications ...
Container orchestration on HPC systems through Kubernetes
Abstract
Containerisation demonstrates its efficiency in application deployment in Cloud Computing. Containers can encapsulate complex programs with their dependencies in isolated environments making applications more portable, hence are being adopted in ...
FLEDGE: Kubernetes Compatible Container Orchestration on Low-Resource Edge Devices
Internet of Vehicles. Technologies and Services Toward Smart Cities
Abstract
In recent years, containers have quickly gained popularity in the cloud, mostly thanks to their scalable, ethereal and isolated nature. Simultaneously, edge devices have become powerful enough to run containerized microservices, while remaining ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MSR '24: Proceedings of the 21st International Conference on Mining Software Repositories

April 2024

788 pages

ISBN:9798400705878

DOI:10.1145/3643991

Chair:
Diomidis Spinellis,
Program Chair:
Alberto Bacchelli,
Program Co-chair:
Eleni Constantinou

Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. National Science Foundation
NSF (National Science Foundation)

Conference

MSR '24

Sponsor:

SIGSOFT

MSR '24: 21st International Conference on Mining Software Repositories

April 15 - 16, 2024

Lisbon, Portugal

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
124
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)32

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Faruqui NRaju NSivakumar SPatel NVengaramkode Bhaskaran SKhanam SBhuiyan T(2025)Gen-Optimizer: A Generative AI Framework for Strategic Business Cost OptimizationComputers10.3390/computers1402005914:2(59)Online publication date: 10-Feb-2025
https://doi.org/10.3390/computers14020059

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten