Chu Chen
ABSTRACT
Certificate parsers, which are critical components of Secure Sockets Layer or Transport Layer Security (SSL/TLS) implementations, parse incomprehensible certificates into comprehensible inputs to certificate validators and humans. Thus, certificate parsers profoundly affect decision-makings of validators and humans, which in turn affect security. To guarantee the correctness of certificate parsers, an approach for search-based differential testing of certificate parsers, namely SBDT, is put forward. SBDT begins with modeling certificate structures, mutation operations, and bounds. Based on the initial model, SBDT searches for the most promising model node and mutation operator that trigger discrepancies, and generates a certificate from the node and operator it finds. Then, SBDT feeds the certificate to certificate parsers, and searches for multiple types of discrepancies after normalizing the results output by parsers. Distinct discrepancies are employed as feedback to update and prune the model. SBDT starts the next iteration from the updated and pruned model, unless all nodes and mutation operators have been pruned due to reaching their upper bounds. Our work has the following contributions: (1) To the best of our knowledge, this is the first time that testing of certificate parsers has been clearly distinguished from testing of certificate validators, which will facilitate accurate testing of certificate parsers and validators; (2) SBDT is the first systematic and efficient approach for differential testing of certificate parsers by searching, updating, and pruning models; and (3) We have implemented an open-source prototype tool of SBDT, and experimental results show that SBDT is effective and efficient in finding new bugs and enhancements of certificate parsers.
- Gergö Barany. 2018. Finding Missed Compiler Optimizations by Differential Testing. In Proceedings of the 27th International Conference on Compiler Construction (CC 2018). Association for Computing Machinery, New York, NY, USA. 82–92. isbn:9781450356442 https://doi.org/10.1145/3178372.3179521
Google Scholar
Digital Library
- Richard Barnes, Martin Thomson, Alfredo Pironti, and Adam Langley. 2015. Deprecating Secure Sockets Layer Version 3.. https://tools.ietf.org/html/rfc7568
Google Scholar
- Chad Brubaker, Suman Jana, Baishakhi Ray, Sarfraz Khurshid, and Vitaly Shmatikov. 2014. Using Frankencerts for Automated Adversarial Testing of Certificate Validation in SSL/TLS Implementations. In Proceedings of the 2014 IEEE Symposium on Security and Privacy (SP ’14). IEEE Computer Society, Washington, DC, USA. 114–129. issn:1081-6011 https://doi.org/10.1109/SP.2014.15
Google ScholarDigital Library
- S. Y. Chau, O. Chowdhury, E. Hoque, H. Ge, A. Kate, C. Nita-Rotaru, and N. Li. 2017. SymCerts: Practical Symbolic Execution for Exposing Noncompliance in X.509 Certificate Validation Implementations. In 2017 IEEE Symposium on Security and Privacy (SP ’17). 503–520. https://doi.org/10.1109/SP.2017.40
Google ScholarCross Ref
- Chu Chen, Cong Tian, Zhenhua Duan, and Liang Zhao. 2018. RFC-Directed Differential Testing of Certificate Validation in SSL/TLS Implementations. In Proceedings of the 40th International Conference on Software Engineering (ICSE 2018). Association for Computing Machinery, New York, NY, USA. 859–870. isbn:9781450356381 https://doi.org/10.1145/3180155.3180226
Google ScholarDigital Library
- Yuting Chen, Ting Su, and Zhendong Su. 2019. Deep Differential Testing of JVM Implementations. In Proceedings of the 41st International Conference on Software Engineering (ICSE 2019). IEEE Press, 1257–1268. https://doi.org/10.1109/ICSE.2019.00127
Google ScholarDigital Library
- Yuting Chen, Ting Su, Chengnian Sun, Zhendong Su, and Jianjun Zhao. 2016. Coverage-Directed Differential Testing of JVM Implementations. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2016). Association for Computing Machinery, New York, NY, USA. 85–99. isbn:9781450342612 https://doi.org/10.1145/2908080.2908095
Google ScholarDigital Library
- Yuting Chen, Ting Su, Chengnian Sun, Zhendong Su, and Jianjun Zhao. 2016. Coverage-Directed Differential Testing of JVM Implementations. SIGPLAN Not., 51, 6 (2016), June, 85–99. issn:0362-1340 https://doi.org/10.1145/2980983.2908095
Google ScholarDigital Library
- Yuting Chen and Zhendong Su.. 2015. Guided Differential Testing of Certificate Validation in SSL/TLS Implementations. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE ’15). ACM, New York, NY, USA. 793–804. isbn:978-1-4503-3675-8 https://doi.org/10.1145/2786805.2786835
Google ScholarDigital Library
- David Cooper, Stefan Santesson, Stephen Farrell, Sharon Boeyen, Russell Housley, and Tim Polk. 2008. Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile. https://tools.ietf.org/html/rfc5280
Google Scholar
- Tim Dierks and Eric Rescorla. 2006. The Transport Layer Security (TLS) Protocol Version 1.1. https://tools.ietf.org/html/rfc4346
Google Scholar
- Tim Dierks and Eric Rescorla. 2008. The Transport Layer Security (TLS) Protocol Version 1.2. https://tools.ietf.org/html/rfc5246
Google Scholar
- Robert B. Evans and Alberto Savoia. 2007. Differential Testing: A New Approach to Change Detection. In The 6th Joint Meeting on European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering: Companion Papers (ESEC-FSE companion 2007). Association for Computing Machinery, New York, NY, USA. 549–552. isbn:9781595938121 https://doi.org/10.1145/1295014.1295038
Google ScholarDigital Library
- Roy Fielding, Jim Gettys, Jeff Mogul, Henrik Frystyk, Larry Masinter, Paul Leach, and Tim Berners-Lee. 1999. Hypertext Transfer Protocol – HTTP/1.1. https://tools.ietf.org/html/rfc2616
Google Scholar
- OpenSSL Software Foundation. 2018. OpenSSL. https://www.openssl.org
Google Scholar
- Alan O. Freier, Philip Karlton, and Paul C. Kocher. 2011. The Secure Sockets Layer (SSL) Protocol Version 3.0. https://tools.ietf.org/html/rfc6101
Google Scholar
- GnuTLS. 2020. GnuTLS does not recognize the DirName and serial of the extension AKI. https://gitlab.com/gnutls/gnutls/-/issues/991
Google Scholar
- GnuTLS. 2020. GnuTLS does not recognize the extension "commonName". https://gitlab.com/gnutls/gnutls/-/issues/989
Google Scholar
- GnuTLS. 2021. A certificate which has no subject public key is parsed by GnuTLS with inconsistent notifications between v3.5.5 and v.3.6.13. https://gitlab.com/gnutls/gnutls/-/issues/1154
Google Scholar
- GnuTLS. 2021. GnuTLS cannot parse Freshest CRL. https://gitlab.com/gnutls/gnutls/-/issues/1156
Google Scholar
- GnuTLS. 2021. GnuTLS cannot parse the ext Netscape Comment. https://gitlab.com/gnutls/gnutls/-/issues/1162
Google Scholar
- GnuTLS. 2021. GnuTLS cannot parse the extension Netscape Cert Type. https://gitlab.com/gnutls/gnutls/-/issues/1159
Google Scholar
- GnuTLS. 2021. GnuTLS cannot parse the extension Policy Constraints. https://gitlab.com/gnutls/gnutls/-/issues/1157
Google Scholar
- GnuTLS. 2021. GnuTLS cannot parse the extension Subject Information Access. https://gitlab.com/gnutls/gnutls/-/issues/1158
Google Scholar
- GnuTLS. 2021. GnuTLS does not completely parse the userNotice in the ext certificatePolicies. https://gitlab.com/gnutls/gnutls/-/issues/1165
Google Scholar
- GnuTLS. 2021. GnuTLS does not parse the ext policyMappings. https://gitlab.com/gnutls/gnutls/-/issues/1166
Google Scholar
- GnuTLS. 2021. GnuTLS rejects a certificate since it parsed the critical extension policyConstraints to unknown ext. https://gitlab.com/gnutls/gnutls/-/issues/1161
Google Scholar
- Muhammad Ali Gulzar, Yongkang Zhu, and Xiaofeng Han. 2019. Perception and Practices of Differential Testing. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2019). IEEE Press, 71–80. https://doi.org/10.1109/ICSE-SEIP.2019.00016
Google ScholarDigital Library
- Jaeyeon Jung, Anmol Sheth, Ben Greenstein, David Wetherall, Gabriel Maganis, and Tadayoshi Kohno. 2008. Privacy Oracle: A System for Finding Application Leaks with Black Box Differential Testing. In Proceedings of the 15th ACM Conference on Computer and Communications Security (CCS 2008). Association for Computing Machinery, New York, NY, USA. 279–288. isbn:9781595938107 https://doi.org/10.1145/1455770.1455806
Google ScholarDigital Library
- Deepak Kumar, Zhengping Wang, Matthew Hyder, Joseph Dickinson, Gabrielle Beck, David Adrian, Joshua Mason, Zakir Durumeric, J. Alex Halderman, and Michael Bailey. 2018. Tracking Certificate Misissuance in the Wild. In 2018 IEEE Symposium on Security and Privacy (SP). 785–798. https://doi.org/10.1109/SP.2018.00015
Google ScholarCross Ref
- Richard B. Langley. 2016. Zmap. https://zmap.io
Google Scholar
- Daniel Lehmann and Michael Pradel. 2018. Feedback-Directed Differential Testing of Interactive Debuggers. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA. 610–620. isbn:9781450355735 https://doi.org/10.1145/3236024.3236037
Google ScholarDigital Library
- LLVM. 2017. libFuzzer. https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer
Google Scholar
- M Marlinspike. 2002. IE SSL Vulnerability. https://www.thoughtcrime.org/ie-ssl-chain.txt
Google Scholar
- M. Marlinspike. 2009. More Tricks for Defeating SSL in Practice. https://www.blackhat.com/presentations/bh-dc-09/Marlinspike/BlackHat-DC- 09-Marlinspike-Defeating-SSL.pdf
Google Scholar
- M. Marlinspike. 2009. New Tricks for Defeating SSL in Practice. https://www.blackhat.com/presentations/bh-usa-09/Marlinspike/BHUSA09-Marlinspike-DefeatSSL-SLIDES.pdf
Google Scholar
- M. Marlinspike. 2009. Null Prefix Attacks against SSL/TLS Certificates. https://www.thoughtcrime.org/papers/null-prefix-attackes.pdf
Google Scholar
- Nikos Mavrogiannopoulos. 2016. GnuTLS. https://www.gnutls.org
Google Scholar
- NIST. 2017. CVE-2016-8495. https://nvd.nist.gov/vuln/detail/CVE-2016-8495
Google Scholar
- Georg Ofenbeck, Tiark Rompf, and Markus Püschel. 2016. RandIR: Differential Testing for Embedded Compilers. In Proceedings of the 2016 7th ACM SIGPLAN Symposium on Scala (SCALA 2016). Association for Computing Machinery, New York, NY, USA. 21–30. isbn:9781450346481 https://doi.org/10.1145/2998392.2998397
Google ScholarDigital Library
- Theofilos Petsios, Adrian Tang, Salvatore Stolfo, Angelos D. Keromytis, and Suman Jana. 2017. NEZHA: Efficient Domain-Independent Differential Testing. In 2017 IEEE Symposium on Security and Privacy (SP ’17). 615–632. https://doi.org/10.1109/SP.2017.27
Google ScholarCross Ref
- Lili Quan, Qianyu Guo, Hongxu Chen, Xiaofei Xie, Xiaohong Li, Yang Liu, and Jing Hu. 2020. SADT: Syntax-Aware Differential Testing of Certificate Validation in SSL/TLS Implementations. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 524–535. issn:2643-1572 https://doi.org/10.1145/3324884.3416552
Google ScholarDigital Library
- Eric Rescorla. 2000. HTTP Over TLS. https://tools.ietf.org/html/rfc2818
Google Scholar
- Eric Rescorla. 2018. The Transport Layer Security (TLS) Protocol Version 1.3. https://tools.ietf.org/html/rfc8446
Google Scholar
- ZMap Team. 2017. ZCertificate. https://github.com/zmap/zcertificate
Google Scholar
- ZMap Team. 2018. ZLint. https://github.com/zmap/zlint
Google Scholar
- Cong Tian, Chu Chen, Zhenhua Duan, and Liang Zhao. 2019. Differential Testing of Certificate Validation in SSL/TLS Implementations: An RFC-Guided Approach. ACM Trans. Softw. Eng. Methodol., 28, 4 (2019), Article 24, Oct., 37 pages. issn:1049-331X https://doi.org/10.1145/3355048
Google ScholarDigital Library
- Yibiao Yang, Yuming Zhou, Hao Sun, Zhendong Su, Zhiqiang Zuo, Lei Xu, and Baowen Xu. 2019. Hunting for Bugs in Code Coverage Tools via Randomized Differential Testing. In Proceedings of the 41st International Conference on Software Engineering (ICSE 2019). IEEE Press, 488–499. https://doi.org/10.1109/ICSE.2019.00061
Google ScholarDigital Library
- Peter E. Yee. 2013. Updates to the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile. https://tools.ietf.org/html/rfc6818
Google Scholar
- ZCertificate. 2021. ZCertificate cannot represent duplicate extensions in JSON output. https://github.com/zmap/zcertificate/issues/23
Google Scholar
- ZCrypto. 2021. Lints: ZCrypto cannot represent the extension name Freshest CRL in JSON output. https://github.com/zmap/zcrypto/issues/244
Google Scholar
- ZCrypto. 2021. ZCrypto cannot parse dnQualifier in JSON output. https://github.com/zmap/zcrypto/issues/252
Google Scholar
- ZCrypto. 2021. ZCrypto cannot represent the extension name Policy Constraints in JSON output. https://github.com/zmap/zcrypto/issues/246
Google Scholar
- ZCrypto. 2021. ZCrypto cannot represent the extension Netscape Cert Type in JSON output. https://github.com/zmap/zcrypto/issues/248
Google Scholar
- ZCrypto. 2021. ZCrypto cannot represent the extension Policy Mappings in JSON output. https://github.com/zmap/zcrypto/issues/250
Google Scholar
- ZCrypto. 2021. ZCrypto cannot represent the extension Subject Information Access in JSON output. https://github.com/zmap/zcrypto/issues/247
Google Scholar
- ZCrypto. 2021. ZCrypto does not parse a authorityKeyId with value “0.”. https://github.com/zmap/zcrypto/issues/256
Google Scholar
- ZCrypto. 2021. ZCrypto does not parse the "critical" of certificatePolicies in JSON output. https://github.com/zmap/zcrypto/issues/255
Google Scholar
- ZCrypto. 2021. ZCrypto does not represent the “critical” of Issuer Alternative Name. https://github.com/zmap/zcrypto/issues/253
Google Scholar
- ZCrypto. 2021. ZCrypto does not represent the “critical” value of keyUsage and basicConstraints in JSON output. https://github.com/zmap/zcrypto/issues/245
Google Scholar
- ZCrypto. 2021. ZCrypto does not represent the ext name inhibitAnyPolicy in JSON output. https://github.com/zmap/zcrypto/issues/254
Google Scholar
- ZCrypto. 2021. ZCrypto parsed critical nameConstraints as non-critical. https://github.com/zmap/zcrypto/issues/251
Google Scholar
- ZLint. 2021. Lints: return detail for e_ext_duplicate_extension. https://github.com/zmap/zlint/issues/550
Google Scholar
- ZLint. 2021. Update e_ext_duplicate_extension error result with Details message. https://github.com/zmap/zlint/issues/549
Google Scholar
Index Terms
- SBDT: Search-Based Differential Testing of Certificate Parsers in SSL/TLS Implementations
Recommendations
Coverage-directed Differential Testing of X.509 Certificate Validation in SSL/TLS Implementations
Secure Sockets Layer (SSL) and Transport Security (TLS) are two secure protocols for creating secure connections over the Internet. X.509 certificate validation is important for security and needs to be performed before an SSL/TLS connection is ...
Differential Testing of Certificate Validation in SSL/TLS Implementations: An RFC-guided Approach
Certificate validation in Secure Sockets Layer or Transport Layer Security protocol (SSL/TLS) is critical to Internet security. Thus, it is significant to check whether certificate validation in SSL/TLS implementations is correctly implemented. With ...
Guided differential testing of certificate validation in SSL/TLS implementations
ESEC/FSE 2015: Proceedings of the 2015 10th Joint Meeting on Foundations of Software EngineeringCertificate validation in SSL/TLS implementations is critical for Internet security. There is recent strong effort, namely frankencert, in automatically synthesizing certificates for stress-testing certificate validation. Despite its early promise, it ...
Comments