Zac Hatfield-Dodds
ABSTRACT
We present Schemathesis, a tool for finding semantic errors and crashes in OpenAPI or GraphQL web APIs through property-based testing. Our evaluation, thirty independent runs of eight tools against sixteen containerized open-source web services, shows that Schemathesis wildly outperforms all previous tools.
It is the only tool to find defects in four targets, finds 1.4× to 4.5× more unique defects than the respectively second-best tool for each remaining target, and is the only tool to handle more than two-thirds of our target services without a fatal internal error.
Our full preprint [5] goes into considerably more detail.
- Cornelius Aschermann, Sergej Schumilo, Ali Abbasi, and Thorsten Holz. 2020. IJON: Exploring Deep State Spaces via Fuzzing. In 2020 IEEE Symposium on Security and Privacy (SP). 1597--1612. Google Scholar
Cross Ref
- Vaggelis Atlidakis, Patrice Godefroid, and Marina Polishchuk. 2020. Checking Security Properties of Cloud Service REST APIs. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE.Google Scholar
- Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming (2000).Google ScholarDigital Library
- Alex Groce, Chaoqiang Zhang, Eric Eide, Yang Chen, and John Regehr. 2012. Swarm Testing. In Proceedings of the 2012 International Symposium on Software Testing and Analysis (ISSTA 2012). 78--88. Google ScholarDigital Library
- Zac Hatfield-Dodds and Dmitry Dygalo. 2021. Deriving Semantics-Aware Fuzzers from Web API Schemas. arXiv:2112.10328 [cs.CR]Google Scholar
- Andreas Löscher and Konstantinos Sagonas. 2018. Automating Targeted Property-Based Testing. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST). 70--80. Google ScholarCross Ref
- David MacIver, Zac Hatfield-Dodds, and Many Contributors. 2019. Hypothesis: A new approach to property-based testing. Journal of Open Source Software 4, 43 (2019), 1891. Google ScholarCross Ref
- David R. MacIver and Alastair F. Donaldson. 2020. Test-Case Reduction via Test-Case Generation: Insights from the Hypothesis Reducer. In 34th European Conference on Object-Oriented Programming (ECOOP).Google Scholar
- Emanuele Viglianisi, Michael Dallago, and Mariano Ceccato. 2020. RESTTESTGEN: Automated Black-Box Testing of RESTful APIs. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST).Google Scholar
Index Terms
- Deriving semantics-aware fuzzers from web API schemas
Recommendations
A Semantics-Enabled Web API Registry
DEXA '11: Proceedings of the 2011 22nd International Workshop on Database and Expert Systems ApplicationsNowadays, Web applications can be quickly developed by combining existing APIs, independently provided by third parties. In this paper we present a semantics-enabled registry for Web APIs and we address the problem of supporting the retrieval and ...
The OWL API: A Java API for OWL ontologies
We present the OWL API, a high level Application Programming Interface (API) for working with OWL ontologies. The OWL API is closely aligned with the OWL 2 structural specification. It supports parsing and rendering in the syntaxes defined in the W3C ...
Bringing Semantics to Web Services with OWL-S
Current industry standards for describing Web Services focus on ensuring interoperability across diverse platforms, but do not provide a good foundation for automating the use of Web Services. Representational techniques being developed for the Semantic ...
Comments