skip to main content
10.1145/3548606.3560661acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article

Non-Distinguishable Inconsistencies as a Deterministic Oracle for Detecting Security Bugs

Published: 07 November 2022 Publication History

Abstract

Security bugs like memory errors are constantly introduced to software programs, and recent years have witnessed an increasing number of reported security bugs. Traditional detection approaches are mainly specification-based---detecting violations against a specified rule as security bugs. This often does not work well in practice because specifications are difficult to specify and generalize, leaving complicated and new types of bugs undetected. Recent research thus leans toward deviation-based detection which finds a substantial number of similar cases and detects deviating cases as potential bugs. This, however, suffers from two other problems. First, it requires enough similar cases to find deviations and thus cannot work for custom code that does not have similar cases. Second, code-similarity analysis is probabilistic and challenging, so the detection can be unreliable. Sometimes, similar cases can normally have deviating behaviors under different contexts.
In this paper, we propose a novel approach for detecting security bugs based on a new concept called Non-Distinguishable Inconsistencies (NDI). The insight is that if two code paths in a function exhibit inconsistent security states (such as being freed or initialized) that are non-distinguishable from the external, such as the callers, there is no way to recover from the inconsistency from the external, which results in a bug. Such an approach has several strengths. First, it is specification-free and thus can support complicated and new types of bugs. Second, it does not require similar cases and by its nature is deterministic. Third, the analysis is practical by minimizing complicated and lengthy data-flow analysis. We implemented NDI and applied it to well-tested programs, including the OpenSSL library, the FreeBSD kernel, the Apache httpd server, and the PHP interpreter. The results show that NDI works for both large and small programs, and it effectively found 51 new bugs, most of which are otherwise missed by the state-of-the-art detection tools.

Supplementary Material

MP4 File (CCS22-fpb510.mp4)
A detailed description for contributions in NDI.

References

[1]
Mansour Ahmadi, Reza Mirzazade farkhani, Ryan Williams, and Long Lu. 2021. Finding Bugs Using Your Own Code: Detecting Functionally-similar yet Inconsistent Code. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2025--2040. https://www.usenix.org/conference/usenixsecurity21/presentation/ahmadi
[2]
Jia-Ju Bai, Julia Lawall, Qiu-Liang Chen, and Shi-Min Hu. 2019. Effective static analysis of concurrency use-after-free bugs in Linux device drivers. In 2019 $$USENIX$$ Annual Technical Conference ($$USENIX$$$$ATC$$ 19). 255--268.
[3]
Jia-Ju Bai, Tuo Li, Kangjie Lu, and Shi-Min Hu. 2021. Static Detection of Unsafe DMA Accesses in Device Drivers. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 1629--1645. https://www.usenix.org/conference/usenixsecurity21/presentation/bai
[4]
Dan Carpenter. 2009. Smatch - the source matcher. http://smatch.sourceforge.
[5]
Navid Emamdoost, Qiushi Wu, Kangjie Lu, and Stephen McCamant. 2021. Detecting kernel memory leaks in specialized modules with ownership reasoning. In Proceedings of the Network and Distributed System Security Symposium.
[6]
Dawson Engler and Ken Ashcraft. 2003. RacerX: effective, static detection of race conditions and deadlocks. ACM SIGOPS operating systems review, Vol. 37, 5 (2003), 237--252.
[7]
Dawson Engler, David Yu Chen, Seth Hallem, Andy Chou, and Benjamin Chelf. 2001. Bugs as deviant behavior: A general approach to inferring errors in systems code. ACM SIGOPS Operating Systems Review, Vol. 35, 5 (2001), 57--72.
[8]
Gang Fan, Rongxin Wu, Qingkai Shi, Xiao Xiao, Jinguo Zhou, and Charles Zhang. 2019. Smoke: scalable path-sensitive memory leak detection for millions of lines of code. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 72--82.
[9]
Josselin Feist, Laurent Mounier, and Marie-Laure Potet. 2014. Statically detecting use after free on binary code. Journal of Computer Virology and Hacking Techniques, Vol. 10, 3 (2014), 211--217.
[10]
Suman Jana, Yuan Jochen Kang, Samuel Roth, and Baishakhi Ray. 2016. Automatically Detecting Error Handling Bugs Using Error Specifications. In USENIX Security Symposium. 345--362.
[11]
Yuede Ji, Lei Cui, and H. Howie Huang. 2021. BugGraph: Differentiating Source-Binary Code Similarity with Graph Triplet-Loss Network. Association for Computing Machinery, New York, NY, USA, 702--715.
[12]
Yuan Kang, Baishakhi Ray, and Suman Jana. 2016. APEx: Automated inference of error specifications for C APIs. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. ACM, 472--482.
[13]
Theodore Kremenek and Dawson R. Engler. 2003. Z-Ranking: Using Statistical Analysis to Counter the Impact of Static Analysis Approximations. In SAS.
[14]
Ted Kremenek, Paul Twohey, Godmar Back, and Andrew Ng. 2006. From Uncertainty to Belief: Inferring the Specification Within. In 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI 06). USENIX Association, Seattle, WA. https://www.usenix.org/conference/osdi-06/uncertainty-belief-inferring-specification-within
[15]
Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Hanchao Qi, and Jie Hu. 2016. VulPecker: An Automated Vulnerability Detection System Based on Code Similarity Analysis. In Proceedings of the 32nd Annual Conference on Computer Security Applications (Los Angeles, California, USA) (ACSAC '16). Association for Computing Machinery, New York, NY, USA, 201--213. https://doi.org/10.1145/2991079.2991102
[16]
Bingchang Liu, Wei Huo, Chao Zhang, Wenchao Li, Feng Li, Aihua Piao, and Wei Zou. 2018. aDiff: Cross-Version Binary Code Similarity Detection with DNN. Association for Computing Machinery, New York, NY, USA, 667--678. https://doi.org/10.1145/3238147.3238199
[17]
Dinghao Liu, Qiushi Wu, Shouling Ji, Kangjie Lu, Zhenguang Liu, Jianhai Chen, and Qinming He. 2021. Detecting Missed Security Operations Through Differential Checking of Object-Based Similar Paths. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS '21). Association for Computing Machinery, 1627--1644. https://doi.org/10.1145/3460120.3485373
[18]
Shen Liu, Gang Tan, and Trent Jaeger. 2017. Ptrsplit: Supporting general pointers in automatic program partitioning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2359--2371.
[19]
Jingbo Lu and Jingling Xue. 2019. Precision-preserving yet fast object-sensitive pointer analysis with partial context sensitivity. Proceedings of the ACM on Programming Languages, Vol. 3, OOPSLA (2019), 1--29.
[20]
Kangjie Lu, Aditya Pakki, and Qiushi Wu. 2019. Detecting Missing-Check Bugs via Semantic- and Context-Aware Criticalness and Constraints Inferences. In 28th USENIX Security Symposium (USENIX Security 19). USENIX Association, 1769--1786.
[21]
Junjie Mao, Yu Chen, Qixue Xiao, and Yuanchun Shi. 2016. RID: finding reference count bugs with inconsistent path pair checking. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems. New York, NY, USA, 531--544.
[22]
Changwoo Min, Sanidhya Kashyap, Byoungyoung Lee, Chengyu Song, and Taesoo Kim. 2015. Cross-checking Semantic Correctness: The Case of Finding File System Bugs. In Proceedings of the 25th ACM Symposium on Operating Systems Principles (SOSP). Monterey, CA.
[23]
Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337--340.
[24]
Ben Niu and Gang Tan. 2014. Modular Control-Flow Integrity. In Proceedings of the 2014 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). Edinburgh, UK.
[25]
Yoann Padioleau, Julia Lawall, René Rydhof Hansen, and Gilles Muller. 2008. Documenting and automating collateral evolutions in Linux device drivers. Acm sigops operating systems review, Vol. 42, 4 (2008), 247--260.
[26]
Aditya Pakki and Kangjie Lu. 2020. Exaggerated Error Handling Hurts! An In-Depth Study and Context-Aware Detection. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery, 1203--1218. https://doi.org/10.1145/3372297.3417256
[27]
Zhou Qingyang, Wu Qiushi, Liu Dinghao, Ji Shouling, and Lu Kangjie. 2022. Non-Distinguishable Inconsistencies as a Deterministic Oracle for Detecting Security Bugs. https://github.com/umnsec/ndi/blob/main/Nondistinguishable_Inconsistencies_as_a_Deterministic_Oracle_for_Detecting_Security_Bugs.pdf
[28]
Lingyun Situ, Linzhang Wang, Yang Liu, Bing Mao, and Xuandong Li. 2018. Vanguard: Detecting Missing Checks for Prognosing Potential Vulnerabilities. In Proceedings of the Tenth Asia-Pacific Symposium on Internetware. ACM, 5.
[29]
Johannes Sp"ath, Karim Ali, and Eric Bodden. 2019. Context-, flow-, and field-sensitive data-flow analysis using synchronized pushdown systems. Proceedings of the ACM on Programming Languages, Vol. 3, POPL (2019), 1--29.
[30]
Lin Tan, Xiaolan Zhang, Xiao Ma, Weiwei Xiong, and Yuanyuan Zhou. 2008. AutoISES: Automatically Inferring Security Specification and Detecting Violations. In USENIX Security Symposium. 379--394.
[31]
Caroline Tice, Tom Roeder, Peter Collingbourne, Stephen Checkoway, Úlfar Erlingsson, Luis Lozano, and Geoff Pike. 2014. Enforcing Forward-Edge Control-Flow Integrity in GCC & LLVM. In USENIX Security Symposium. 941--955.
[32]
Victor van der Veen, Dennis Andriesse, Manolis Stamatogiannakis, Xi Chen, Herbert Bos, and Cristiano Giuffrdia. 2017. The dynamics of innocent flesh on the bone: Code reuse ten years later. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 1675--1689.
[33]
Shuai Wang and Dinghao Wu. 2017. In-Memory Fuzzing for Binary Code Similarity Analysis. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (Urbana-Champaign, IL, USA) (ASE 2017). IEEE Press, 319--330.
[34]
Wenwen Wang, Kangjie Lu, and Pen-Chung Yew. 2018. Check it Again: Detecting Lacking-Recheck Bugs in OS Kernels. In Proceedings of the 25th ACM Conference on Computer and Communications Security (CCS). Toronto, Canada.
[35]
Westley Weimer and George C Necula. 2005. Mining temporal specifications for error detection. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 461--476.
[36]
Qiushi Wu, Yang He, Stephen McCamant, and Kangjie Lu. 2020. Precisely Characterizing Security Impact in a Flood of Patches via Symbolic Rule Comparison. In Proceedings of the 27th Annual Network and Distributed System Security Symposium (NDSS'20).
[37]
Qian Wu, Guangtai Liang, Qianxiang Wang, Tao Xie, and Hong Mei. 2011. Iterative mining of resource-releasing specifications. In 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, 233--242.
[38]
Qiushi Wu, Aditya Pakki, Navid Emamdoost, Stephen McCamant, and Kangjie Lu. 2021. Understanding and Detecting Disordered Error Handling with Precise Function Pairing. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2041--2058. https://www.usenix.org/conference/usenixsecurity21/presentation/wu-qiushi
[39]
Yang Xiao, Bihuan Chen, Chendong Yu, Zhengzi Xu, Zimu Yuan, Feng Li, Binghong Liu, Yang Liu, Wei Huo, Wei Zou, and Wenchang Shi. 2020. MVP: Detecting Vulnerabilities using Patch-Enhanced Vulnerability Signatures. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 1165--1182.
[40]
Xiaojun Xu, Chang Liu, Qian Feng, Heng Yin, Le Song, and Dawn Xiaodong Song. 2017. Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detection. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017).
[41]
Hua Yan, Yulei Sui, Shiping Chen, and Jingling Xue. 2018. Spatio-Temporal Context Reduction: A Pointer-Analysis-Based Static Approach for Detecting Use-after-Free Vulnerabilities. In Proceedings of the 40th International Conference on Software Engineering. Association for Computing Machinery, 327--337.
[42]
Zeping Yu, Rui Cao, Qiyi Tang, Sen Nie, Junzhou Huang, and Shi Wu. 2020. Order Matters: Semantic-Aware Neural Networks for Binary Code Similarity Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 01 (Apr. 2020), 1145--1152. https://doi.org/10.1609/aaai.v34i01.5466
[43]
Insu Yun, Changwoo Min, Xujie Si, Yeongjin Jang, Taesoo Kim, and Mayur Naik. 2016. APISan: Sanitizing API Usages through Semantic Cross-Checking. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 363--378.
[44]
Yizhuo Zhai, Yu Hao, Zheng Zhang, Weiteng Chen, Guoren Li, Zhiyun Qian, Chengyu Song, Manu Sridharan, Srikanth V Krishnamurthy, Trent Jaeger, et al. 2022. Progressive Scrutiny: Incremental Detection of UBI bugs in the Linux Kernel. In Proceedings of the 29th Annual Network and Distributed System Security Symposium (NDSS'22). io

Cited By

View all
  • (2024)Inference of error specifications and bug detection using structural similaritiesProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699006(1885-1902)Online publication date: 14-Aug-2024

Index Terms

  1. Non-Distinguishable Inconsistencies as a Deterministic Oracle for Detecting Security Bugs

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CCS '22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security
      November 2022
      3598 pages
      ISBN:9781450394505
      DOI:10.1145/3548606
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 November 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. deterministic bug detection
      2. non-distinguishable inconsistencies
      3. static analysis

      Qualifiers

      • Research-article

      Funding Sources

      • National Science Foundation

      Conference

      CCS '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

      Upcoming Conference

      CCS '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)81
      • Downloads (Last 6 weeks)3
      Reflects downloads up to 01 Mar 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Inference of error specifications and bug detection using structural similaritiesProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699006(1885-1902)Online publication date: 14-Aug-2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media