skip to main content
10.1145/3530019.3530031acmotherconferencesArticle/Chapter ViewAbstractPublication PageseaseConference Proceedingsconference-collections
research-article

Exploring Security Vulnerabilities in Competitive Programming: An Empirical Study

Published: 13 June 2022 Publication History

Abstract

Insecure code leading to software vulnerabilities can result in damages of the order of millions of dollars, and in critical systems, the loss of life. Hence, developing secure systems free of exploitable vulnerabilities has been a thrust area of research in recent years. Understanding developers’ approach towards vulnerabilities in their code can pave the way for improvements in insecure coding practices. Recent studies have explored online Q&A forums, open-source code repositories, and other code information sources to gain important insights into the pervasiveness of security vulnerabilities. However, to the best of our knowledge, competitive programming (CP) data, a rich source of information about coding practices, has not been explored from the perspective of insecure coding practices. The evaluation and assessment of coding practices used in CP is particularly intriguing because it has become a key player in developer recruitment in recent times. In this paper, we make one of the first attempts to draw the attention of the community to the emergent concern of insecure coding practices in CP. We use static analysis tools to identify the prevalence and nature of vulnerabilities in a large amount of CP data (6.1 million submissions) obtained from a top-rated CP platform, CodeChef, and find that 34.2% of submissions contain vulnerabilities. We observe that many programmers consistently follow insecure coding practices and most of the detected vulnerabilities are characterized by security standards (CWE, CVSS) based on real-world software.

References

[1]
Bushra Aloraini, Meiyappan Nagappan, Daniel M German, Shinpei Hayashi, and Yoshiki Higo. 2019. An empirical study of security warnings from static application security testing tools. Journal of Systems and Software 158 (2019), 110427.
[2]
Andrei Arusoaie, Stefan Ciobâca, Vlad Craciun, Dragos Gavrilut, and Dorel Lucanu. 2017. A Comparison of Open-Source Static Analysis Tools for Vulnerability Detection in C/C++ Code. In 2017 19th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC). 161–168. https://doi.org/10.1109/SYNASC.2017.00035
[3]
Wei Bai, Omer Akgul, and Michelle L Mazurek. 2019. A qualitative investigation of insecure code propagation from online forums. In 2019 IEEE Cybersecurity Development (SecDev). IEEE, 34–48.
[4]
Miguel Brito and Celestino Gonçalves. 2019. Codeflex: A Web-based Platform for Competitive Programming. In 2019 14th Iberian Conference on Information Systems and Technologies (CISTI). 1–6. https://doi.org/10.23919/CISTI.2019.8760776
[5]
Uriel Ferreira Campos, Guilherme Smethurst, Joao Pedro Moraes, Rodrigo Bonifácio, and Gustavo Pinto. 2019. Mining rule violations in javascript code snippets. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 195–199.
[6]
Lingwei Chen, Shifu Hou, Yanfang Ye, Thirimachos Bourlai, Shouhuai Xu, and Liang Zhao. 2019. iTrustSO: an intelligent system for automatic detection of insecure code snippets in stack overflow. In 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 1097–1104.
[7]
Daniel Coore and Daniel Fokum. 2019. Facilitating Course Assessment with a Competitive Programming Platform. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 449–455. https://doi.org/10.1145/3287324.3287511
[8]
Tania Di Mascio, Luigi Laura, and Marco Temperini. 2018. A framework for personalized competitive programming training. In 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET). IEEE, 1–8.
[9]
Ehsan Firouzi, Ashkan Sami, Foutse Khomh, and Gias Uddin. 2020. On the use of C# Unsafe Code Context: An Empirical Study of Stack Overflow. In Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–6.
[10]
Felix Fischer, Konstantin Böttinger, Huang Xiao, Christian Stransky, Yasemin Acar, Michael Backes, and Sascha Fahl. 2017. Stack overflow considered harmful? the impact of copy&paste on android application security. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 121–136.
[11]
Kevin Fu, Tadayoshi Kohno, Daniel Lopresti, Elizabeth Mynatt, Klara Nahrstedt, Shwetak Patel, Debra Richardson, and Ben Zorn. 2020. Safety, security, and privacy threats posed by accelerating trends in the internet of things. arXiv preprint arXiv:2008.00017(2020).
[12]
Tiago Espinha Gasiba, Ulrike Lechner, Maria Pinto-Albuquerque, and Daniel Mendez. 2021. Is Secure Coding Education in the Industry Needed? An Investigation Through a Large Scale Survey. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). IEEE, 241–252.
[13]
Hamza Ghani, Jesus Luna, and Neeraj Suri. 2013. Quantitative assessment of software vulnerabilities based on economic-driven security metrics. In 2013 International Conference on Risks and Security of Internet and Systems (CRiSIS). IEEE, 1–8.
[14]
Hristina Gulabovska and Zoltán Porkoláb. 2019. Survey on Static Analysis Tools of Python Programs. In SQAMIA.
[15]
Mohammadreza Hazhirpasand, Mohammad Ghafari, Stefan Krüger, Eric Bodden, and Oscar Nierstrasz. 2019. The impact of developer experience in using Java cryptography. In 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). IEEE, 1–6.
[16]
Mohammadreza Hazhirpasand, Mohammad Ghafari, and Oscar Nierstrasz. 2020. Java Cryptography Uses in the Wild. In Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–6.
[17]
Hyunji Hong, Seunghoon Woo, and Heejo Lee. 2021. Dicos: Discovering Insecure Code Snippets from Stack Overflow Posts by Leveraging User Discussions. In Annual Computer Security Applications Conference. 194–206.
[18]
Nicolas Huaman, Bennet von Skarczinski, Christian Stransky, Dominik Wermke, Yasemin Acar, Arne Dreißigacker, and Sascha Fahl. 2021. A Large-Scale Interview Study on Information Security in and Attacks against Small and Medium-sized Enterprises. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 1235–1252. https://www.usenix.org/conference/usenixsecurity21/presentation/huaman
[19]
Emanuele Iannone, Roberta Guadagni, Filomena Ferrucci, Andrea De Lucia, and Fabio Palomba. 2022. The Secret Life of Software Vulnerabilities: A Large-Scale Empirical Study. IEEE Transactions on Software Engineering(2022).
[20]
Ratnakar Kumar, Nitasha Hasteer, and Jean-Paul Van Belle. 2018. Evaluating Factors Influencing Contestant Behavior in Competitive Software Development. In 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE, 20–25.
[21]
Jessica Lam, Elias Fang, Majed Almansoori, Rahul Chatterjee, and Adalbert Gerald Soosai Raj. 2022. Identifying Gaps in the Secure Programming Knowledge and Skills of Students. In In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education.
[22]
Ben Lazarine, Sagar Samtani, Mark Patton, Hongyi Zhu, Steven Ullman, Benjamin Ampel, and Hsinchun Chen. 2020. Identifying Vulnerable GitHub Repositories and Users in Scientific Cyberinfrastructure: An Unsupervised Graph Embedding Approach. In 2020 IEEE International Conference on Intelligence and Security Informatics (ISI). IEEE, 1–6.
[23]
Triet Le Le Huynh Minh, Roland Croft, David Hin, and Muhammad Ali Ali Babar. 2021. A Large-scale Study of Security Vulnerability Support on Developer Q&A Websites. In Evaluation and Assessment in Software Engineering. 109–118.
[24]
Bingchang Liu, Liang Shi, Zhuhua Cai, and Min Li. 2012. Software Vulnerability Discovery Techniques: A Survey. In 2012 Fourth International Conference on Multimedia Information Networking and Security. 152–156. https://doi.org/10.1109/MINES.2012.202
[25]
Tamara Lopez, Helen Sharp, Thein Tun, Arosha Bandara, Mark Levine, and Bashar Nuseibeh. 2019. ” Hopefully We Are Mostly Secure”: Views on Secure Code in Professional Practice. In 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). IEEE, 61–68.
[26]
Gary McGraw. 2006. Software Security: Building Security In. In 2006 17th International Symposium on Software Reliability Engineering. 24–27. https://doi.org/10.1109/ISSRE.2006.43
[27]
Sarah Meldrum, Sherlock A Licorish, Caitlin A Owen, and Bastin Tony Roy Savarimuthu. 2020. Understanding stack overflow code quality: A recommendation of caution. Science of Computer Programming 199 (2020), 102516.
[28]
Michael Meli, Matthew R McNiece, and Bradley Reaves. 2019. How Bad Can It Git? Characterizing Secret Leakage in Public GitHub Repositories. In NDSS.
[29]
Na Meng, Stefan Nagy, Danfeng Yao, Wenjie Zhuang, and Gustavo Arango Argoty. 2018. Secure coding practices in java: Challenges and vulnerabilities. In Proceedings of the 40th International Conference on Software Engineering. 372–383.
[30]
Fabiola Moyon, Kristian Beckers, Sebastian Klepper, Philipp Lachberger, and Bernd Bruegge. 2018. Towards continuous security compliance in agile software development at scale. In 2018 IEEE/ACM 4th International Workshop on Rapid Continuous Software Engineering (RCoSE). IEEE, 31–34.
[31]
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden. 2016. Jumping through hoops: Why do Java developers struggle with cryptography APIs?. In Proceedings of the 38th International Conference on Software Engineering. 935–946.
[32]
Prashant R Nair. 2020. Increasing Employability of Indian Engineering Graduates through Experiential Learning Programs and Competitive Programming: Case Study. Procedia Computer Science 172 (2020), 831–837.
[33]
Yu Nong, Haipeng Cai, Pengfei Ye, Li Li, and Feng Chen. 2021. Evaluating and comparing memory error vulnerability detectors. Information and Software Technology 137 (2021), 106614.
[34]
Ivan Pashchenko, Riccardo Scandariato, Antonino Sabetta, and Fabio Massacci. 2021. Secure Software Development in the Era of Fluid Multi-party Open Software and Services. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE, 91–95.
[35]
José D’Abruzzo Pereira, Naghmeh Ivaki, and Marco Vieira. 2021. Characterizing Buffer Overflow Vulnerabilities in Large C/C++ Projects. IEEE Access 9(2021), 142879–142892.
[36]
José D’Abruzzo Pereira and Marco Vieira. 2020. On the Use of Open-Source C/C++ Static Analysis Tools in Large Projects. In 2020 16th European Dependable Computing Conference (EDCC). IEEE, 97–102.
[37]
Akond Rahman, Effat Farhana, and Nasif Imtiaz. 2019. Snakes in paradise?: Insecure python-related coding practices in stack overflow. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). IEEE, 200–204.
[38]
Rahul Telang and Sunil Wattal. 2007. An Empirical Analysis of the Impact of Software Vulnerability Announcements on Firm Stock Price. IEEE Transactions on Software Engineering 33, 8 (2007), 544–557. https://doi.org/10.1109/TSE.2007.70712
[39]
Morteza Verdi, Ashkan Sami, Jafar Akhondali, Foutse Khomh, Gias Uddin, and Alireza Karami Motlagh. 2020. An empirical study of c++ vulnerabilities in crowd-sourced code examples. IEEE Transactions on Software Engineering(2020).
[40]
Yanfang Ye, Shifu Hou, Lingwei Chen, Xin Li, Liang Zhao, Shouhuai Xu, Jiabin Wang, and Qi Xiong. 2018. Icsd: An automatic system for insecure code snippet detection in stack overflow over heterogeneous information network. In Proceedings of the 34th Annual Computer Security Applications Conference. 542–552.
[41]
Tianyi Zhang, Ganesha Upadhyaya, Anastasia Reinhardt, Hridesh Rajan, and Miryung Kim. 2018. Are code examples on an online Q&A forum reliable?: a study of API misuse on stack overflow. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 886–896.

Cited By

View all
  • (2024)ICode - An Unified Competitive Coding Profile Platform2024 International Conference on Emerging Smart Computing and Informatics (ESCI)10.1109/ESCI59607.2024.10497289(1-5)Online publication date: 5-Mar-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering
June 2022
466 pages
ISBN:9781450396134
DOI:10.1145/3530019
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Competitive Programming
  2. Empirical Study
  3. Security Vulnerabilities
  4. Software Security
  5. Static Analysis

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

EASE 2022

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)45
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)ICode - An Unified Competitive Coding Profile Platform2024 International Conference on Emerging Smart Computing and Informatics (ESCI)10.1109/ESCI59607.2024.10497289(1-5)Online publication date: 5-Mar-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media