skip to main content
research-article

Blindspots in Python and Java APIs Result in Vulnerable Code

Published: 26 April 2023 Publication History

Abstract

Blindspots in APIs can cause software engineers to introduce vulnerabilities, but such blindspots are, unfortunately, common. We study the effect APIs with blindspots have on developers in two languages by replicating a 109-developer, 24-Java-API controlled experiment. Our replication applies to Python and involves 129 new developers and 22 new APIs. We find that using APIs with blindspots statistically significantly reduces the developers’ ability to correctly reason about the APIs in both languages, but that the effect is more pronounced for Python. Interestingly, for Java, the effect increased with complexity of the code relying on the API, whereas for Python, the opposite was true. This suggests that Python developers are less likely to notice potential for vulnerabilities in complex code than in simple code, whereas Java developers are more likely to recognize the extra complexity and apply more care, but are more careless with simple code. Whether the developers considered API uses to be more difficult, less clear, and less familiar did not have an effect on their ability to correctly reason about them. Developers with better long-term memory recall were more likely to correctly reason about APIs with blindspots, but short-term memory, processing speed, episodic memory, and memory span had no effect. Surprisingly, professional experience and expertise did not improve the developers’ ability to reason about APIs with blindspots across both languages, with long-term professionals with many years of experience making mistakes as often as relative novices. Finally, personality traits did not significantly affect the Python developers’ ability to reason about APIs with blindspots, but less extroverted and more open developers were better at reasoning about Java APIs with blindspots. Overall, our findings suggest that blindspots in APIs are a serious problem across languages, and that experience and education alone do not overcome that problem, suggesting that tools are needed to help developers recognize blindspots in APIs as they write code that uses those APIs.

References

[1]
Y. Acar, M. Backes, S. Fahl, S. Garfinkel, D. Kim, M. L. Mazurek, and C. Stransky. 2017. Comparing the usability of cryptographic APIs. In IEEE Symposium on Security and Privacy (SP’17). 154–171. DOI:DOI:
[2]
Y. Acar, M. Backes, S. Fahl, D. Kim, M. L. Mazurek, and C. Stransky. 2016. You get where you’re looking for: The impact of information sources on code security. In IEEE Symposium on Security and Privacy (SP’16). 289–305. DOI:DOI:
[3]
Y. Acar, M. Backes, S. Fahl, D. Kim, M. L. Mazurek, and C. Stransky. 2017. How internet resources might be helping you develop faster but less securely. IEEE Secur. Priv. 15, 2 (Mar.2017), 50–60. DOI:DOI:
[4]
Y. Acar, S. Fahl, and M. L. Mazurek. 2016. You are not your developer, either: A research agenda for usable security and privacy research beyond end users. In IEEE Cybersecurity Development (SecDev’16). 3–8. DOI:DOI:
[5]
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. 2018. A reductions approach to fair classification. In International Conference on Machine Learning (ICML’18), Vol. PMLR 80. 60–69.
[6]
Rico Angell, Brittany Johnson, Yuriy Brun, and Alexandra Meliou. 2018. Themis: Automatically testing software for discrimination. In Demonstrations Track at the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’18). 871–875. DOI:DOI:
[7]
John Anvik, Lyndon Hiew, and Gail C. Murphy. 2006. Who should fix this bug? In International Conference on Software Engineering (ICSE’06). 361–370. DOI:DOI:
[8]
Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason I Hong, and Lorrie Faith Cranor. 2014. The privacy and security behaviors of smartphone app developers. In Workshop on Usable Security. Internet Society.
[9]
Emery D. Berger, Celeste Hollenbeck, Petr Maj, Olga Vitek, and Jan Vitek. 2019. On the impact of programming languages on code quality: A reproduction study. ACM Trans. Program. Lang. Syst. 41, 4 (Oct.2019). DOI:DOI:
[10]
Ivan Beschastnikh, Jenny Abrahamson, Yuriy Brun, and Michael D. Ernst. 2011. Synoptic: Studying logged behavior with inferred models. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering Tool Demonstration Track (ESEC/FSE’11). 448–451. DOI:DOI:
[11]
Ivan Beschastnikh, Yuriy Brun, Jenny Abrahamson, Michael D. Ernst, and Arvind Krishnamurthy. 2013. Unifying FSM-inference algorithms through declarative specification. In International Conference on Software Engineering (ICSE’13). 252–261. DOI:DOI:
[12]
Ivan Beschastnikh, Yuriy Brun, Jenny Abrahamson, Michael D. Ernst, and Arvind Krishnamurthy. 2015. Using declarative specification to improve the understanding, extensibility, and comparison of model-inference algorithms. IEEE Trans. Softw. Eng. 41, 4 (Ap.2015), 408–428. DOI:DOI:
[13]
Ivan Beschastnikh, Yuriy Brun, Michael D. Ernst, and Arvind Krishnamurthy. 2014. Inferring models of concurrent systems from logs of their behavior with CSight. In International Conference on Software Engineering (ICSE’14). 468–479. DOI:DOI:
[14]
Ivan Beschastnikh, Yuriy Brun, Michael D. Ernst, Arvind Krishnamurthy, and Thomas E. Anderson. 2011. Mining temporal invariants from partially ordered logs. ACM SIGOPS Oper. Syst. Rev. 45, 3 (Dec.2011), 39–46. DOI:DOI:
[15]
Ivan Beschastnikh, Yuriy Brun, Sigurd Schneider, Michael Sloan, and Michael D. Ernst. 2011. Leveraging existing instrumentation to automatically infer invariant-constrained models. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’11). 267–277. DOI:DOI:
[16]
Ivan Beschastnikh, Perry Liu, Albert Xing, Patty Wang, Yuriy Brun, and Michael D. Ernst. 2020. Visualizing distributed system executions. ACM Trans. Softw. Eng. Methodol. 29, 2 (Mar.2020), 9:1–9:38. DOI:DOI:
[17]
Ivan Beschastnikh, Patty Wang, Yuriy Brun, and Michael D. Ernst. 2016. Debugging distributed systems. Commun. ACM 59, 8 (Aug.2016), 32–37. DOI:DOI:
[18]
Tom Britton, Lisa Jeng, Graham Carver, Paul Cheak, and Tomer Katzenellenbogen. 2013. Reversible Debugging Software. Technical Report. University of Cambridge, Judge Business School.
[19]
Yuriy Brun, George Edwards, Jae young Bang, and Nenad Medvidovic. 2011. Smart redundancy for distributed computation. In International Conference on Distributed Computing Systems (ICDCS’11). 665–676. DOI:DOI:
[20]
Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2011. Crystal: Precise and unobtrusive conflict warnings. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering Tool Demonstration Track (ESEC/FSE’11). 444–447. DOI:DOI:
[21]
Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2011. Proactive detection of collaboration conflicts. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’11). 168–178. DOI:DOI:
[22]
Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2013. Early detection of collaboration conflicts and risks. IEEE Trans. Softw. Eng. 39, 10 (Oct.2013), 1358–1375. DOI:DOI:
[23]
Yuriy Brun and Nenad Medvidovic. 2007. An architectural style for solving computationally intensive problems on large networks. In Software Engineering for Adaptive and Self-Managing Systems (SEAMS’07). DOI:DOI:
[24]
Yuriy Brun and Nenad Medvidovic. 2007. Fault and adversary tolerance as an emergent property of distributed systems’ software architectures. In 2nd International Workshop on Engineering Fault Tolerant Systems (EFTS’07). 38–43. DOI:DOI:
[25]
Yuriy Brun and Nenad Medvidovic. 2012. Keeping data private while computing in the cloud. In 5th International Conference on Cloud Computing (CLOUD’12). 285–294. DOI:DOI:
[26]
Yuriy Brun and Nenad Medvidovic. 2013. Entrusting private computation and data to untrusted networks. IEEE Trans. Depend. Secure Comput. 10, 4 (July/Aug.2013), 225–238. DOI:DOI:
[27]
Yuriy Brun and Alexandra Meliou. 2018. Software fairness. In New Ideas and Emerging Results Track at the 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’18). 754–759. DOI:DOI:
[28]
Justin Cappos, Yanyan Zhuang, Daniela Oliveira, Marissa Rosenthal, and Kuo-Chuan Yeh. 2014. Vulnerabilities as blind spots in developer’s heuristic-based decision-making processes. In New Security Paradigms Workshop (NSPW’14). 53–62. DOI:DOI:
[29]
Michael Coblenz, Whitney Nelson, Jonathan Aldrich, Brad Myers, and Joshua Sunshine. 2017. Glacier: Transitive class immutability for Java. In International Conference on Software Engineering (ICSE’17). 496–506. DOI:DOI:
[30]
Michael Coblenz, Joshua Sunshine, Jonathan Aldrich, Brad Myers, Sam Weber, and Forrest Shull. 2016. Exploring language support for immutability. In International Conference on Software Engineering (ICSE’16). 736–747. DOI:DOI:
[31]
Common Weakness Enumeration 2011. Common Weakness Enumeration (CWE)/SANS Top 25 Most Dangerous Software Errors. Retrieved from.
[32]
Paul T. Costa and Robert R. MacCrae. 1992. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI): Professional Manual. Psychological Assessment Resources, Incorporated.
[33]
Barthélémy Dagenais and Martin P. Robillard. 2010. Creating and evolving developer documentation: Understanding the decisions of open source contributors. In ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’10). 127–136. DOI:DOI:
[34]
Valentin Dallmeier, Nikolai Knopp, Christoph Mallon, Sebastian Hack, and Andreas Zeller. 2010. Generating test cases for specification mining. In International Symposium on Software Testing and Analysis (ISSTA’10). 85–96. DOI:DOI:
[35]
Valentin Dallmeier, Christian Lindig, and Andreas Zeller. 2005. Lightweight defect localization for Java. In European Conference on Object Oriented Programming (ECOOP’05). 528–550. DOI:DOI:
[36]
Zoltan Dienes. 2014. Using Bayes to get the most out of non-significant results. Front. Psychol. 5 (2014), 781:1–781:17. DOI:DOI:
[37]
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Innovations in Theoretical Computer Science Conference (ITCS’12). 214–226.
[38]
Brian Ellis, Jeffrey Stylos, and Brad Myers. 2007. The factory pattern in API design: A usability evaluation. In International Conference on Software Engineering (ICSE’07). 302–312. DOI:DOI:
[39]
Emily First and Yuriy Brun. 2022. Diversity-driven Automated Formal Verification. In 44th International Conference on Software Engineering (ICSE’22). 749–761. DOI:DOI:
[40]
Emily First, Yuriy Brun, and Arjun Guha. 2020. TacTok: Semantics-aware proof synthesis. Proc. ACM. Program. Lang. Object-Orient. Program. Syst. Lang. Applic. 4 (Nov.2020), 231:1–231:31. DOI:DOI:
[41]
F. Fischer, K. B\(\ddot{o}\)ttinger, H. Xiao, C. Stransky, Y. Acar, M. Backes, and S. Fahl. 2017. Stack overflow considered harmful? The impact of copy paste on Android application security. In IEEE Symposium on Security and Privacy (SP’17). 121–136. DOI:DOI:
[42]
Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness testing: Testing software for discrimination. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’17). 498–510. DOI:DOI:
[43]
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Longman Publishing Co., Inc.
[44]
Richard C. Gershon, Molly V. Wagster, Hugh C. Hendrie, Nathan A. Fox, Karon F. Cook, and Cindy J. Nowinski. 2013. NIH toolbox for assessment of neurological and behavioral function. Neurology 80, 11 Supplement 3 (2013), S2–S6.
[45]
Carlo Ghezzi, Mauro Pezzè, Michele Sama, and Giordano Tamburrelli. 2014. Mining behavior models from user-intensive web applications. In ACM/IEEE International Conference on Software Engineering (ICSE’14). 277–287. DOI:DOI:
[46]
Stephen Giguere, Blossom Metevier, Yuriy Brun, Bruno Castro da Silva, Philip S. Thomas, and Scott Niekum. 2022. Fairness guarantees under demographic shift. In 10th International Conference on Learning Representations (ICLR’22). Retrieved from https://openreview.net/forum?id=wbPObLm6ueA.
[47]
Dan Gopstein, Jake Iannacone, Yu Yan, Lois DeLong, Yanyan Zhuang, Martin K.-C. Yeh, and Justin Cappos. 2017. Understanding misunderstandings in source code. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’17). 129–139. DOI:DOI:
[48]
GraphicsMagick 2017. GraphicsMagick 1.4 Heap-based Buffer Overflow Vulnerability. Retrieved from.
[49]
M. Green and M. Smith. 2016. Developers are not the enemy!: The need for usable security APIs. IEEE Secur. Priv. 14, 5 (Sept.2016), 40–46. DOI:DOI:
[50]
Nina Grgic-Hlaca, Elissa M. Redmiles, Krishna P. Gummadi, and Adrian Weller. 2018. Human perceptions of fairness in algorithmic decision making: A case study of criminal risk prediction. In World Wide Web Conference (WWW’18). 903–912. DOI:DOI:
[51]
Henry L. Roediger III and K. Andrew DeSoto. 2014. Confidence and memory: Assessing positive and negative correlations. Memory 22, 1 (2014), 76–91. DOI:DOI:
[52]
Oliver P. John and Sanjay Srivastava. 1999. The big five trait taxonomy: History, measurement, and theoretical perspectives. Handb. Personal.: Theor. Res. 2, 1999 (1999), 102–138.
[53]
Brittany Johnson, Jesse Bartola, Rico Angell, Katherine Keith, Sam Witty, Stephen J. Giguere, and Yuriy Brun. 2020. Fairkit, fairkit, on the wall, who’s the fairest of them all? Supporting data scientists in training fair models. CoRR abs/2012.09951 (2020).
[54]
Brittany Johnson and Yuriy Brun. 2022. Fairkit-learn: A fairness evaluation and comparison toolkit. In Demonstrations Track at the 44th International Conference on Software Engineering (ICSE’22). 70–74. DOI:DOI:
[55]
Brittany Johnson, Yuriy Brun, and Alexandra Meliou. 2020. Causal Testing: Understanding Defects’ Root Causes. In International Conference on Software Engineering (ICSE’20). 87–99. DOI:DOI:
[56]
Brittany Johnson, Rahul Pandita, Justin Smith, Denae Ford, Sarah Elder, Emerson Murphy-Hill, Sarah Heckman, and Caitlin Sadowski. 2016. A cross-tool communication study on program analysis tool notifications. In ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’16). 73–84. DOI:DOI:
[57]
James A. Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of test information to assist fault localization. In International Conference on Software Engineering (ICSE’02). 467–477. DOI:DOI:
[58]
Herb Krasner. 2020. The Cost of Poor Software Quality in the US: A 2020 Report. Retrieved from https://www.it-cisq.org/pdf/CPSQ-2020-report.pdf.
[59]
Ivo Krka, Yuriy Brun, and Nenad Medvidovic. 2014. Automatic mining of specifications from invocation traces and method invariants. In ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE’14). 178–189. DOI:DOI:
[60]
Margie E. Lachman, Stefan Agrigoroaei, Patricia A. Tun, and Suzanne L. Weaver. 2014. Monitoring cognitive functioning: Psychometric properties of the brief test of adult cognition by telephone. Assessment 21, 4 (2014). DOI:DOI:
[61]
Ben Liblit, Mayur Naik, Alice X. Zheng, Alex Aiken, and Michael I. Jordan. 2005. Scalable statistical bug isolation. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’05). 15–26. DOI:DOI:
[62]
Thomas J. McCabe. 1976. A complexity measure. IEEE Trans. Softw. Eng. 2, 4 (July1976), 308–320. DOI:DOI:
[63]
Joe McManus and Sandy Shrum. 2015. SEI CERT Oracle Coding Standard for Java. Retrieved from.
[64]
Blossom Metevier, Stephen Giguere, Sarah Brockman, Ari Kobren, Yuriy Brun, Emma Brunskill, and Philip Thomas. 2019. Offline contextual bandits with high probability fairness guarantees. In Annual Conference on Neural Information Processing Systems (NeurIPS), Advances in Neural Information Processing Systems 32. 14893–14904.
[65]
Emerson Murphy-Hill, Da Young Lee, Gail C. Murphy, and Joanna McGrenere. 2015. How do users discover new tools in software development and beyond?Comput. Supp. Coop. Work 24, 5 (01 Oct.2015), 389–422. DOI:DOI:
[66]
Brad A. Myers and Jeffrey Stylos. 2016. Improving API usability. Commun. ACM 59, 6 (May2016), 62–69. DOI:DOI:
[67]
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden. 2016. Jumping through hoops: Why do Java developers struggle with cryptography APIs? In International Conference on Software Engineering (ICSE’16). 935–946. DOI:DOI:
[68]
National Vulnerability Database 1999. National Vulnerability Database. Retrieved from.
[69]
Tony Ohmann, Michael Herzberg, Sebastian Fiss, Armand Halbert, Marc Palyart, Ivan Beschastnikh, and Yuriy Brun. 2014. Behavioral resource-aware model inference. In IEEE/ACM International Conference on Automated Software Engineering (ASE’14). 19–30. DOI:DOI:
[70]
Daniela Oliveira, Marissa Rosenthal, Nicole Morin, Kuo-Chuan Yeh, Justin Cappos, and Yanyan Zhuang. 2014. It’s the psychology stupid: How heuristics explain software vulnerabilities and how priming can illuminate developer’s blind spots. In Annual Computer Security Applications Conference (ACSAC’14). 296–305. DOI:DOI:
[71]
Daniela Seabra Oliveira, Tian Lin, Muhammad Sajidur Rahman, Rad Akefirad, Donovan Ellis, Eliany Perez, Rahul Bobhate, Lois A. DeLong, Justin Cappos, Yuriy Brun, and Natalie C. Ebner. 2018. API blindspots: Why experienced developers write vulnerable code. In USENIX Symposium on Usable Privacy and Security (SOUPS’18). 315–328. Retrieved from https://www.usenix.org/system/files/conference/soups2018/soups2018-oliveira.pdf.
[72]
Marten Oltrogge, Yasemin Acar, Sergej Dechand, Matthew Smith, and Sascha Fahl. 2015. To pin or not to pin helping app developers bullet proof their TLS connections. In USENIX Conference on Security Symposium (SEC’15). USENIX Association, 239–254.
[73]
Open Web Application Security Project 2013. The Open Web Application Security Project (OWASP) Top 10 Most Critical Web Application Security Risks. Retrieved from.
[74]
H. Orman. 2003. The Morris worm: A fifteen-year perspective. IEEE Secur. Priv. 1, 5 (Sept.2003), 35–43. DOI:DOI:
[75]
James Parker, Michael Hicks, Andrew Ruef, Michelle L. Mazurek, Dave Levin, Daniel Votipka, Piotr Mardziel, and Kelsey R. Fulton. 2020. Build it, break it, fix it: Contesting secure development. ACM Trans. Priv. Secur. 23, 2 (Apr.2020), 10:1–10:36. DOI:DOI:
[76]
Andreas Poller, Laura Kocksch, Katharina Kinder-Kurlanda, and Felix Anand Epp. 2016. First-time security audits as a turning point?: Challenges for security practices in an industry software development team. In SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI AE’16). 1288–1294. DOI:DOI:
[77]
Andreas Poller, Laura Kocksch, Sven Türpe, Felix Anand Epp, and Katharina Kinder-Kurlanda. 2017. Can security become a routine?: A study of organizational change in an agile software development group. In ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW’17). 2489–2503. DOI:DOI:
[78]
Muhammad Sajidur Rahman. 2016. An Empirical Case Study on Stack Overflow to Explore Developers’ Security Challenges. Masters Report. Retrieved from http://krex.k-state.edu/dspace/handle/2097/34563.
[79]
M. P. Robillard. 2009. What makes APIs hard to learn? Answers from developers. IEEE Softw. 26, 6 (Nov.2009), 27–34. DOI:DOI:
[80]
Martin P. Robillard and Robert Deline. 2011. A field study of API learning obstacles. Empir. Softw. Eng. 16, 6 (Dec.2011), 703–732. DOI:DOI:
[81]
Timothy A. Salthouse and Kenneth A. Prill. 1987. Inferences about age impairments in inferential reasoning. Psychol. Aging 2, 1 (1987). DOI:DOI:
[82]
Alex Sanchez-Stern, Emily First, Timothy Zhou, Zhanna Kaufman, Yuriy Brun, and Talia Ringer. 2022. Passport: Improving automated formal verification using identifiers. CoRR abs/2204.10370 (2022).
[83]
Anita Sarma, Zahra Noroozi, and André van der Hoek. 2003. Palantír: Raising awareness among configuration management workspaces. In International Conference on Software Engineering (ICSE’03). 444–454. DOI:DOI:
[84]
Anita Sarma, David F. Redmiles, and André van der Hoek. 2012. Palantír: Early detection of development conflicts arising from parallel code changes. IEEE Trans. Softw. Eng. 38, 4 (2012), 889–908. DOI:DOI:
[85]
K. Warner Schaie. 1996. Intellectual Development in Adulthood: The Seattle Longitudinal Study. Cambridge University Press, New York, NY.
[86]
Secure Coding Guidelines 2022. Secure Coding Guidelines for Java SE, Oracle. Retrieved from.
[87]
Security Focus Vulnerability Database 2021. Security Focus Vulnerability Database, Accenture. Retrieved from.
[88]
Security Vulnerabilities. 2017. Security Vulnerabilities (SQL Injection), MITRE Corporation. Retrieved from.
[89]
Konrad Slind and Michael Norrish. 2008. A brief overview of HOL4. In International Conference on Theorem Proving in Higher Order Logics (TPHOLs’08). 28–32. DOI:DOI:
[90]
Stack Overflow 2008. Stack Overflow: A Q/A Site for Professional and Enthusiast Programmers, Stack Overflow. Retrieved from.
[91]
State of Software Security. 2016. State of Software Security, Veracode. Retrieved from.
[92]
Andreas Stefik and Susanna Siebert. 2013. An empirical investigation into programming language syntax. Trans. Comput. Educ. 13, 4 19 (Nov.2013). DOI:DOI:
[93]
Jeffrey Stylos and Steven Clarke. 2007. Usability implications of requiring parameters in objects’ constructors. In International Conference on Software Engineering (ICSE’07). 529–539. DOI:DOI:
[94]
Symantec. 2017. Symantec Internet Security Threat Report. Retrieved from.
[95]
The Coq Development Team. 2017. Coq, v.8.7. Retrieved from https://coq.inria.fr.
[96]
Philip S. Thomas, Bruno Castro da Silva, Andrew G. Barto, Stephen Giguere, Yuriy Brun, and Emma Brunskill. 2019. Preventing undesirable behavior of intelligent machines. Science 366, 6468 (22 Nov.2019), 999–1004. DOI:DOI:
[97]
Florian Tramer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2017. FairTest: Discovering unwarranted associations in data-driven applications. In IEEE European Symposium on Security and Privacy (EuroS&P’17).
[98]
Patricia A. Tun and Margie E. Lachman. 2006. Telephone assessment of cognitive function in adulthood: The brief test of adult cognition by telephone. Age Ageing 35, 6 (2006), 629–632.
[99]
Tommi Unruh, Bhargava Shastry, Malte Skoruppa, Federico Maggi, Konrad Rieck, Jean-Pierre Seifert, and Fabian Yamaguchi. 2017. Leveraging flawed tutorials for seeding large-scale web vulnerability discovery. In 11th USENIX Workshop on Offensive Technologies (WOOT’17). USENIX Association. Retrieved from https://www.usenix.org/conference/woot17/workshop-program/presentation/unruh.
[100]
Dirk van der Linden, Pauline Anthonysamy, Bashar Nuseibeh, Thein T. Tun, Marian Petre, Mark Levine, John Towse, and Awais Rashid. 2020. Schrödinger’s security: Opening the box on app developers’ security rationale. In International Conference on Software Engineering (ICSE’20). DOI:DOI:
[101]
Daniel Votipka, Kelsey R. Fulton, James Parker, Matthew Hou, Michelle L. Mazurek, and Michael Hicks. 2020. Understanding security mistakes developers make: Qualitative analysis from Build It, Break It, Fix It. In 29th USENIX Security Symposium (USENIX Security). Retrieved from https://www.usenix.org/conference/usenixsecurity20/presentation/votipka-understanding.
[102]
D. Votipka, R. Stevens, E. Redmiles, J. Hu, and M. Mazurek. 2018. Hackers vs. testers: A comparison of software vulnerability discovery processes. In IEEE Symposium on Security and Privacy (SP’18). 134–151. DOI:DOI:
[103]
Zhiyuan Wan, Xin Xia, David Lo, Jiachi Chen, Xiapu Luo, and Xiaohu Yang. 2021. Smart contract security: A practitioners’ perspective. In International Conference on Software Engineering (ICSE’21).
[104]
Sam Weber, Michael Coblenz, Brad Myers, Jonathan Aldrich, and Joshua Sunshine. 2017. Empirical studies on the security and usability impact of immutability. In IEEE Cybersecurity Development (SecDev’17). 50–53. DOI:DOI:
[105]
Chamila Wijayarathna and Nalin Asanka Gamagedara Arachchilage. 2019. Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation. Inf. Softw. Technol. 115 (2019), 5–19. DOI:DOI:
[106]
Jim Witschey, Shundan Xiao, and Emerson Murphy-Hill. 2014. Technical and personal factors influencing developers’ adoption of security tools. In ACM Workshop on Security Information Workers (SIW’14). 23–26. DOI:DOI:
[107]
Jim Witschey, Olga Zielinska, Allaire Welk, Emerson Murphy-Hill, Chris Mayhorn, and Thomas Zimmermann. 2015. Quantifying developers’ adoption of security tools. In Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’15). 260–271. DOI:DOI:
[108]
Shundan Xiao, Jim Witschey, and Emerson Murphy-Hill. 2014. Social influences on secure development tool adoption: Why security tools spread. In ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW’14). 1095–1106. DOI:DOI:
[109]
Jing Xie, Heather Lipford, and Bei-Tseng Chu. 2012. Evaluating interactive support for secure programming. In SIGCHI Conference on Human Factors in Computing Systems (CHI’12). 2707–2716. DOI:DOI:
[110]
Jing Xie, Heather Richter Lipford, and Bill Chu. 2011. Why do programmers make security errors? In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 161–164.
[111]
Kaiyu Yang and Jia Deng. 2019. Learning to prove theorems via interacting with proof assistants. In International Conference on Machine Learning (ICML’19). Retrieved from http://proceedings.mlr.press/v97/yang19a/yang19a.pdf.

Cited By

View all
  • (2024)Security Analysis of Large Language Models on API Misuse Programming RepairInternational Journal of Intelligent Systems10.1155/2024/71357652024Online publication date: 1-Jan-2024
  • (2024)Methods and Benchmark for Detecting Cryptographic API Misuses in PythonIEEE Transactions on Software Engineering10.1109/TSE.2024.337718250:5(1118-1129)Online publication date: May-2024
  • (2024)Secure Cryptography Usage in Software Development: A Systematic Literature Review2024 12th International Conference in Software Engineering Research and Innovation (CONISOFT)10.1109/CONISOFT63288.2024.00036(218-227)Online publication date: 28-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology
ACM Transactions on Software Engineering and Methodology  Volume 32, Issue 3
May 2023
937 pages
ISSN:1049-331X
EISSN:1557-7392
DOI:10.1145/3594533
  • Editor:
  • Mauro Pezzè
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2023
Online AM: 19 November 2022
Accepted: 18 October 2022
Revised: 29 June 2022
Received: 01 December 2021
Published in TOSEM Volume 32, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Software vulnerabilities
  2. Java
  3. Python
  4. APIs
  5. API blindspots

Qualifiers

  • Research-article

Funding Sources

  • National Science Foundation

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)253
  • Downloads (Last 6 weeks)23
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Security Analysis of Large Language Models on API Misuse Programming RepairInternational Journal of Intelligent Systems10.1155/2024/71357652024Online publication date: 1-Jan-2024
  • (2024)Methods and Benchmark for Detecting Cryptographic API Misuses in PythonIEEE Transactions on Software Engineering10.1109/TSE.2024.337718250:5(1118-1129)Online publication date: May-2024
  • (2024)Secure Cryptography Usage in Software Development: A Systematic Literature Review2024 12th International Conference in Software Engineering Research and Innovation (CONISOFT)10.1109/CONISOFT63288.2024.00036(218-227)Online publication date: 28-Oct-2024
  • (2024)“C”ing the light – assessing code comprehension in novice programmers using C code patternsComputer Science Education10.1080/08993408.2024.2317079(1-25)Online publication date: 15-Feb-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media