Abstract
With the rise of the web as a dominant application platform, web security vulnerabilities are of increasing concern. Ideally, the web application development process would detect and correct these vulnerabilities before they are released to the public. This research aims to quantify the effectiveness of software developers at security code review as well as determine the variation in effectiveness among web developers. We hired 30 developers to conduct a manual code review of a small web application. The web application supplied to developers had seven known vulnerabilities, including three different types: Cross-Site Scripting, Cross-Site Request Forgery, and SQL Injection. Our findings include: (1) none of the subjects found all confirmed vulnerabilities, (2) more experience does not necessarily mean that the reviewer will be more accurate or effective, and (3) reports of false vulnerabilities were significantly correlated with reports of valid vulnerabilities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
TopSite: 10 Best Outsourcing Websites, http://www.topsite.com/best/outsourcing
OWASP Foundation: Code Review Metrics (2010), https://www.owasp.org/index.php/Code_Review_Metrics
Baca, D., Petersen, K., Carlsson, B., Lundberg, L.: Static code analysis to detect software security vulnerabilities—does experience matter? In: International Conference on Availability, Reliability and Security, ARES 2009, pp. 804–810. IEEE (2009)
Fagan, M.E.: Design and Code Inspections to Reduce Errors in Program Development. IBM Systems Journal 15(3), 182–211 (1976)
McCarthy, P., Porter, A., Siy, H., Votta Jr., L.G.: An Experiment to Assess Cost-Benefits of Inspection Meetings and Their Alternatives: A Pilot Study. In: Proceedings of the 3rd International Software Metrics Symposium, pp. 100–111 (March 1996)
Biffl, S.: Analysis of the Impact of Reading Technique and Inspector Capability on Individual Inspection Performance. In: Proceedings of the Seventh Asia-Pacific Software Engineering Conference (APSEC), pp. 136–145 (2000)
Hatton, L.: Predicting the Total Number of Faults Using Parallel Code Inspections (May 2005), http://www.leshatton.org/2005/05/total-number-of-faults-using-parallel-code-inspections/
Hatton, L.: Testing the Value of Checklists in Code Inspections. IEEE Software 25(4), 82–88 (2008)
Albayrak, O., Davenport, D.: Impact of Maintainability Defects on Code Inspections. In: Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 50:1–50:4 (2010)
Ferreira, A., Machado, R., Costa, L., Silva, J., Batista, R., Paulk, M.: An Approach to Improving Software Inspections Performance. In: 2010 IEEE International Conference on Software Maintenance (ICSM), pp. 1–8 (September 2010)
Bau, J., Bursztein, E., Gupta, D., Mitchell, J.: State of the Art: Automated Black-Box Web Application Vulnerability Testing. In: 2010 IEEE Symposium on Security and Privacy, pp. 332–345 (May 2010)
Huang, Y.W., Yu, F., Hang, C., Tsai, C.H., Lee, D.T., Kuo, S.Y.: Securing Web Application Code by Static Analysis and Runtime Protection. In: Proceedings of the 13th International Conference on the World Wide Web, pp. 40–52 (2004)
Kals, S., Kirda, E., Kruegel, C., Jovanovic, N.: SecuBat: A Web Vulnerability Scanner. In: Proceedings of the 15th International Conference on the World Wide Web, pp. 247–256 (2006)
Jovanovic, N., Kruegel, C., Kirda, E.: Pixy: A Static Analysis Tool for Detecting Web Application Vulnerabilities. In: IEEE Symposium on Security and Privacy, pp. 263–268 (May 2006)
Wassermann, G., Su, Z.: Sound and Precise Analysis of Web Applications for Injection Vulnerabilities. In: Proceedings of the 2007 ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 32–41 (June 2007)
Lam, M.S., Martin, M., Livshits, B., Whaley, J.: Securing Web Applications With Static and Dynamic Information Flow Tracking. In: Proceedings of the 2008 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-Based Program Manipulation, pp. 3–12 (2008)
Kieyzun, A., Guo, P., Jayaraman, K., Ernst, M.: Automatic Creation of SQL Injection and Cross-Site Scripting Attacks. In: 31st IEEE International Conference on Software Engineering, pp. 199–209 (May 2009)
Basili, V., Selby, R.: Comparing the Effectiveness of Software Testing Strategies. IEEE Transactions on Software Engineering SE-13(12), 1278–1296 (1987)
Jones, C.: Software Defect-Removal Efficiency. IEEE Computer 29(4), 94–95 (1996)
Finifter, M., Wagner, D.: Exploring the Relationship Between Web Application Development Tools and Security. In: Proceedings of the 2nd USENIX Conference on Web Application Development. USENIX (June 2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Edmundson, A., Holtkamp, B., Rivera, E., Finifter, M., Mettler, A., Wagner, D. (2013). An Empirical Study on the Effectiveness of Security Code Review. In: Jürjens, J., Livshits, B., Scandariato, R. (eds) Engineering Secure Software and Systems. ESSoS 2013. Lecture Notes in Computer Science, vol 7781. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36563-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-36563-8_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36562-1
Online ISBN: 978-3-642-36563-8
eBook Packages: Computer ScienceComputer Science (R0)