skip to main content
10.1145/3551349.3559509acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaseConference Proceedingsconference-collections
short-paper

ASTOR: An Approach to Identify Security Code Reviews

Published: 05 January 2023 Publication History

Abstract

During code reviews, software developers often raise security concerns if they find any. Ignoring such concerns can bring a severe impact on the performance of a software product. This risk can be reduced if we can automatically identify such code reviews that trigger security concerns so that we can perform additional scrutiny from the security experts. Therefore, the objective of this study is to develop an automated tool to identify code reviews that trigger security concerns.
With this goal, I developed an approach named ASTOR, where I combine two separate deep learning-based classifiers– (i) using code review comments and (ii) using the corresponding code context, and make an ensemble using Logistic Regression. Based on stratified ten-fold cross-validation, the best ensemble model achieves the F1-score of 79.8% with an accuracy of 88.4% to automatically identify code reviews that raise security concerns.

References

[1]
[n.d.]. Clang Static Analyzer. https://clang-analyzer.llvm.org/. [Online; accessed on July 22, 2022].
[2]
Toufique Ahmed, Amiangshu Bosu, Anindya Iqbal, and Shahram Rahimi. 2017. SentiCR: a customized sentiment analysis tool for code review interactions. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 106–111.
[3]
Alberto Bacchelli and Christian Bird. 2013. Expectations, outcomes, and challenges of modern code review. In Proceedings of the 2013 international conference on software engineering. IEEE Press, 712–721.
[4]
Amiangshu Bosu, Jeffrey C. Carver, Hafiz Munawar, Patrick Hilley, and Derek Janni. 2014. Identifying the Characteristics of Vulnerable Code Changes: an Empirical Study. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 257–268.
[5]
Amiangshu Bosu, Michaela Greiler, and Christian Bird. 2015. Characteristics of useful code reviews: An empirical study at microsoft. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. IEEE, 146–156.
[6]
Felivel Camilo, Andrew Meneely, and Meiyappan Nagappan. 2015. Do bugs foreshadow vulnerabilities?: a study of the Chromium project. In Proceedings of the 12th Working Conference on Mining Software Repositories. IEEE Press, 269–279.
[7]
Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.
[8]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).
[9]
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, 2020. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155(2020).
[10]
Gerrit Code Review. [n.d.]. Gerrit Code Review - REST API. https://gerrit-review.googlesource.com/Documentation/rest-api.html. [Online; accessed Septempber 1, 2022].
[11]
Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 18, 5-6 (2005), 602–610.
[12]
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, 2020. Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366(2020).
[13]
Rie Johnson and Tong Zhang. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 562–570.
[14]
J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.
[15]
G. McGraw. 2006. Software security: building security in. volume 1. Addison-Wesley Professional(2006).
[16]
Andrew Meneely, Harshavardhan Srinivasan, Ayemi Musa, Alberto Rodriguez Tejeda, Matthew Mokary, and Brian Spates. 2013. When a patch goes bad: Exploring the properties of vulnerability-contributing commits. In 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. IEEE, 65–74.
[17]
Nuthan Munaiah and Andrew Meneely. 2016. Vulnerability severity scoring and bounties: why the disconnect?. In Proceedings of the 2nd International Workshop on Software Analytics. ACM, 8–14.
[18]
Rajshakhar Paul, Asif Kamal Turzo, and Amiangshu Bosu. 2021. Why security defects go unnoticed during code reviews? a case-control study of the chromium os project. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 1373–1385.
[19]
Celia Paulsen. 2018. Glossary of key information security terms. Technical Report. National Institute of Standards and Technology.
[20]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825–2830.
[21]
Dewayne E Perry, Harvey P Siy, and Lawrence G Votta. 2001. Parallel changes in large-scale software development: an observational case study. ACM Transactions on Software Engineering and Methodology (TOSEM) 10, 3(2001), 308–337.
[22]
Peter C Rigby and Christian Bird. 2013. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 202–212.
[23]
Taro Yamane. 1967. Statistics: An introductory analysis.Technical Report.

Cited By

View all
  • (2023)Formal Methods and Validation Techniques for Ensuring Automotive Systems SecurityInformation10.3390/info1412066614:12(666)Online publication date: 18-Dec-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering
October 2022
2006 pages
ISBN:9781450394758
DOI:10.1145/3551349
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. code review
  2. deep learning
  3. discussion
  4. security
  5. vulnerability

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Conference

ASE '22

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)3
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Formal Methods and Validation Techniques for Ensuring Automotive Systems SecurityInformation10.3390/info1412066614:12(666)Online publication date: 18-Dec-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media