ABSTRACT
Developers often use the C preprocessor to handle variability and portability. However, many researchers and practitioners criticize the use of preprocessor directives because of their negative effect on code understanding, maintainability, and error proneness. This negative effect may lead to configuration-related code weaknesses, which appear only when we enable or disable certain configuration options. A weakness is a type of mistake in software that, in proper conditions, could contribute to the introduction of vulnerabilities within that software. Configuration-related code weaknesses may be harder to detect and fix than weaknesses that appear in all configurations, because variability increases complexity. To address this problem, we propose a sampling-based white-box technique to detect configuration-related weaknesses in configurable systems. To evaluate our technique, we performed an empirical study with 24 popular highly configurable systems that make heavy use of the C preprocessor, such as Apache Httpd and Libssh. Using our technique, we detected 57 configuration-related weaknesses in 16 systems. In total, we found occurrences of the following five kinds of weaknesses: 30 memory leaks, 10 uninitialized variables, 9 null pointer dereferences, 6 resource leaks, and 2 buffer overflows. The corpus of these weaknesses is a valuable source to better support further research on configuration-related code weaknesses.
- 2020. Cppcheck Design. http://cppcheck.sourceforge.net/.Google Scholar
- Iago Abal, Claus Brabrand, and Andrzej Wasowski. 2014. 42 Variability Bugs in the Linux Kernel: A Qualitative Analysis. In Proceedings of the International Conference on Automated Software Engineering. 421--432.Google ScholarDigital Library
- Iago Abal, Jean Melo, Stefan Stănciulescu, Claus Brabrand, Márcio Ribeiro, and Andrzej Wasowski. 2018. Variability Bugs in Highly Configurable Systems: A Qualitative Analysis. Transactions on Software Engineering and Methodology 26, 3 (2018), 10:1--10:34.Google Scholar
- Ira D. Baxter. 1992. Design maintenance systems. Commun. ACM 35, 4 (1992), 73--89.Google ScholarDigital Library
- Ira D. Baxter and Michael Mehlich. 2001. Preprocessor conditional removal by simple partial evaluation. In Proceedings of the Working Conference on Reverse Engineering. IEEE, Germany, 281--290.Google Scholar
- Michael D. Bond and Kathryn S McKinley. 2008. Tolerating memory leaks. In Proceedings of the Object-Oriented Programming Systems Languages and Applications. 109--126.Google Scholar
- Larissa Braz, Rohit Gheyi, Melina Mongiovi, Márcio Ribeiro, Flávio Medeiros, and Leopoldo Teixeira. 2016. A Change-centric Approach to Compile Configurable Systems with #Ifdefs. In Proceedings of the 15th International Conference on Generative Programming: Concepts & Experiences. 109--119.Google ScholarDigital Library
- Larissa Braz, Rohit Gheyi, Melina Mongiovi, Márcio Ribeiro, Flávio Medeiros, Leopoldo Teixeira, and Sabrina Souto. 2018. A change-aware per-file analysis to compile configurable systems with #ifdefs. Computer Languages, Systems & Structures 54 (2018), 427--450.Google ScholarDigital Library
- Renée Bryce and Charles Colbourn. 2006. Prioritized interaction testing for pairwise coverage with seeding and constraints. Information and Software Technology 48, 10 (2006), 960--970.Google ScholarCross Ref
- Al Danial. 2020. CLOC. http://cloc.sourceforge.net/.Google Scholar
- Christian Dietrich, Reinhard Tartler, Wolfgang Schroder-Preikschat, and Daniel Lohmann. 2012. A robust approach for variability extraction from the Linux build system. In Proceedings of the Software Product-Line Conference. 21--30.Google ScholarDigital Library
- Michael Ernst, Greg Badros, and David Notkin. 2002. An Empirical Analysis of C Preprocessor Use. Transactions on Software Engineering 28, 12 (2002), 1146--1170.Google ScholarDigital Library
- Gabriel Ferreira, Momin Malik, Christian Kästner, Jürgen Pfeffer, and Sven Apel. 2016. Do #ifdefs influence the occurrence of vulnerabilities? An empirical study of the Linux kernel. In Proceedings of the International Systems and Software Product Line Conference. 65--73.Google ScholarDigital Library
- Matthew Finifter, Devdatta Akhawe, and David Wagner. 2013. An empirical study of vulnerability rewards programs. In Proceedings of the USENIX Conference on Security. 273--288.Google Scholar
- Stefan Frei, Dominik Schatzmann, Bernhard Plattner, and Brian Trammell. 2010. Modeling the security ecosystem - the dynamics of (In)security. Springer US, 79--106.Google Scholar
- Alejandra Garrido and Ralph Johnson. 2002. Challenges of Refactoring C Programs. In Proceedings of the International Workshop on Principles of Software Evolution. 6--14.Google ScholarDigital Library
- Alejandra Garrido and Ralph Johnson. 2003. Refactoring C with Conditional Compilation. In Proceedings of the International Conference on Automated Software Engineering. 323--326.Google ScholarDigital Library
- Alejandra Garrido and Ralph Johnson. 2005. Analyzing Multiple Configurations of a C Program. In Proceedings of the International Conference on Software Maintenance. 379--388.Google ScholarDigital Library
- Brady Garvin and Myra Cohen. 2011. Feature Interaction Faults Revisited: An Exploratory Study. In Proceedings of the International Symposium on Software Reliability Engineering. 90--99.Google ScholarDigital Library
- Brady Garvin, Myra Cohen, and Matthew Dwyer. 2011. Using Feature Locality: Can We Leverage History to Avoid Failures During Reconfiguration?. In Proceedings of the Workshop on Assurances for Self-adaptive Systems.Google ScholarDigital Library
- Paul Gazzillo and Robert Grimm. 2012. SuperC: parsing all of C by taming the preprocessor. In Proceedings of the Programming Language Design and Implementation. 323--334.Google ScholarDigital Library
- Axel Halin, Alexandre Nuttinck, Mathieu Acher, Xavier Devroey, Gilles Perrouin, and Benoit Baudry. 2017. Test them all, is it worth it? A ground truth comparison of configuration sampling strategies. arXiv preprint arXiv:1710.07980 (2017).Google Scholar
- Kyo Kang, Sholom Cohen, James Hess, William Novak, and Spencer Peterson. 1990. Feature-Oriented Domain Analysis Feasibility Study. Technical Report. Carnegie Mellon University.Google Scholar
- Christian Kastner and Sven Apel. 2009. Virtual Separation of Concerns -A Second Chance for Preprocessors. Journal of Object Technology 8, 6 (2009), 59--78.Google ScholarCross Ref
- Christian Kastner, Paolo Giarrusso, Tillmann Rendel, Sebastian Erdweg, Klaus Ostermann, and Thorsten Berger. 2011. Variability-Aware Parsing in the Presence of Lexical Macros and Conditional Compilation. In Proceedings of the Object-Oriented Programming Systems Languages and Applications. 805--824.Google ScholarDigital Library
- Jorg Liebig, Sven Apel, Christian Lengauer, Christian Kastner, and Michael Schulze. 2010. An analysis of the variability in forty preprocessor-based software product lines. In Proceedings of the International Conference on Software Engineering. 105--114.Google ScholarDigital Library
- Jorg Liebig, Christian Kastner, and Sven Apel. 2011. Analyzing the discipline of preprocessor annotations in 30 million lines of C code. In Proceedings of the International Conference on Aspect-Oriented Software Development. 191--202.Google ScholarDigital Library
- Jorg Liebig, Alexander von Rhein, Christian Kastner, Sven Apel, Jens Dorre, and Christian Lengauer. 2013. Scalable Analysis of Variable Software. In Proceedings of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering. 81--91.Google ScholarDigital Library
- Flávio Medeiros, Christian Kastner, Márcio Ribeiro, Rohit Gheyi, and Sven Apel. 2016. A Comparison of 10 Sampling Algorithms for Configurable Systems. In Proceedings of the International Conference on Software Engineering. 643--654.Google ScholarDigital Library
- Flávio Medeiros, Christian Kastner, Márcio Ribeiro, Sarah Nadi, and Rohit Gheyi. 2015. The Love/Hate Relationship with the C Preprocessor: An Interview Study. In Proceedings of the European Conference on Object-Oriented Programming. 999--1022.Google Scholar
- Flávio Medeiros, Márcio Ribeiro, and Rohit Gheyi. 2013. Investigating Preprocessor-Based Syntax Errors. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences. 75--84.Google ScholarDigital Library
- Flávio Medeiros, Iran Rodrigues, Márcio Ribeiro, Leopoldo Teixeira, and Rohit Gheyi. 2015. An Empirical Study on Configuration-Related Issues: Investigating Undeclared and Unused Identifiers. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences. 35--44.Google ScholarDigital Library
- Mitre. 2019. Top 25 Most Dangerous Software Errors. http://cwe.mitre.org/top25/.Google Scholar
- Mitre. 2020. Uninitialized Variable. https://cwe.mitre.org/data/definitions/457.html.Google Scholar
- Mitre. 2020. Weaknesses. https://cwe.mitre.org/documents/glossary/index.html#Weakness.Google Scholar
- Austin Mordahl, Jeho Oh, Ugur Koc, Shiyi Wei, and Paul Gazzillo. 2019. An empirical study of real-world variability bugs detected by variability-oblivious tools. In Proceedings of the Foundations of Software Engineering. 50--61.Google ScholarDigital Library
- Raphael Muniz, Larissa Braz, Rohit Gheyi, Wilkerson Andrade, Baldoino Fonseca, and Márcio Ribeiro. 2018. A Qualitative Analysis of Variability Weaknesses in Configurable Systems with #Ifdefs. In Proceedings of the International Workshop on Variability Modelling of Software-Intensive Systems. 51--58.Google ScholarDigital Library
- Sarah Nadi and Richard Holt. 2014. The Linux kernel: A case study of build system variability. Journal of Software: Evolution and Process 26, 8 (2014), 730--746.Google ScholarDigital Library
- Changhai Nie and Hareton Leung. 2011. A Survey of Combinatorial Testing. Computing Surveys 43, 2 (2011), 11:1--11:29.Google Scholar
- Sebastian Oster, Florian Markert, and Philipp Ritter. 2010. Automated Incremental Pairwise Testing of Software Product Lines. In Software Product Lines: Going Beyond, Jan Bosch and Jaejoon Lee (Eds.). Lecture Notes in Computer Science, Vol. 6287. 196--210.Google Scholar
- OWASP. 2020. Buffer Overflow. https://owasp.org/www-community/vulnerabilities/Buffer_Overflow.Google Scholar
- OWASP. 2020. Memory Leak. https://owasp.org/www-community/vulnerabilities/Memory_leak.Google Scholar
- OWASP. 2020. Null Pointer Dereference. https://owasp.org/www-community/vulnerabilities/Null_Dereference.Google Scholar
- OWASP. 2020. Resource Leak. https://owasp.org/www-community/vulnerabilities/Unreleased_Resource.Google Scholar
- Nicolas Palix, Gael Thomas, Suman Saha, Christophe Calves, Julia Lawall, and Gilles Muller. 2011. Faults in Linux: Ten Years Later. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems. 305--318.Google ScholarDigital Library
- Leonardo Passos, Jianmei Guo, Leopoldo Teixeira, Krzysztof Czarnecki, Andrzej Wasowski, and Paulo Borba. 2013. Coevolution of Variability Models and Related Artifacts: A Case Study from the Linux Kernel. In Proceedings of the International Software Product Line Conference. 91--100.Google ScholarDigital Library
- Gilles Perrouin, Sagar Sen, and Jacques Klein. 2010. Automated and Scalable T-wise Test Case Generation Strategies for Product Lines. In Proceeding of the International Conference on Software Testing, Verification and Validation. 459--468.Google ScholarDigital Library
- Sabrina Souto, Marcelo d'Amorim, and Rohit Gheyi. 2017. Balancing Soundness and Efficiency for Practical Testing of Configurable Systems. In Proceedings of the International Conference on Software Engineering. 632--642.Google ScholarDigital Library
- Henry Spencer and Geoff Collyer. 1992. Ifdef Considered Harmful, or Portability Experience with C News. In Proceendings of the USENIX Annual Technical Conference. USENIX Association.Google Scholar
- Reinhard Tartler, Christian Dietrich, Julio Sincero, Wolfgang Schroder-Preikschat, and Daniel Lohmann. 2014. Static Analysis of Variability in System Software: The 90,000 #ifdefs Issue. In USENIX Annual Technical Conference. 421--432.Google Scholar
- Our Team. 2020. Supplementary website. https://sbesweaknesses.github.io/.Google Scholar
- David Wheeler. 2020. FlawFinder. https://www.dwheeler.com/flawfinder/.Google Scholar
Index Terms
An Empirical Study on Configuration-Related Code Weaknesses
Recommendations
A Qualitative Analysis of Variability Weaknesses in Configurable Systems with #ifdefs
VAMOS '18: Proceedings of the 12th International Workshop on Variability Modelling of Software-Intensive SystemsA number of critical configurable systems are implemented using #ifdefs, such as Linux. Some tools and strategies are proposed to avoid these directives. However, these systems still have weaknesses, leading to vulnerable code, and may impact millions ...
An empirical study on configuration-related issues: investigating undeclared and unused identifiers
GPCE 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesThe variability of configurable systems may lead to configuration-related issues (i.e., faults and warnings) that appear only when we select certain configuration options. Previous studies found that issues related to configurability are harder to ...
An empirical study on configuration-related issues: investigating undeclared and unused identifiers
GPCE '15The variability of configurable systems may lead to configuration-related issues (i.e., faults and warnings) that appear only when we select certain configuration options. Previous studies found that issues related to configurability are harder to ...
Comments