  Fuzzing -- whether generating or mutating inputs -- has found many bugs and
security vulnerabilities in a wide range of domains. Stateful and highly
structured web APIs present significant challenges to traditional fuzzing
techniques, as execution feedback is usually limited to a response code instead
of code coverage and vulnerabilities of interest include silent
information-disclosure in addition to explicit errors.
  Our tool, Schemathesis, derives structure- and semantics-aware fuzzers from
web API schemas in the OpenAPI or GraphQL formats, using property-based testing
tools. Derived fuzzers can be incorporated into unit-test suites or run
directly, with or without end-user customisation of data generation and
semantic checks.
  We construct the most comprehensive evaluation of web API fuzzers to date,
running eight fuzzers against sixteen real-world open source web services.
OpenAPI schemas found in the wild have a long tail of rare features and complex
structures. Of the tools we evaluated, Schemathesis was the only one to handle
more than two-thirds of our target services without a fatal internal error.
Schemathesis finds 1.4 times to 4.5 times more unique defects than the
respectively second-best fuzzer for each target, and is the only fuzzer to find
defects in four targets.
