  Coverage analysis is widely used but can suffer from high overhead. This
overhead is especially acute in the context of Python, which is already
notoriously slow (a recent study observes a roughly 30x slowdown vs. native
code). We find that the state-of-the-art coverage tool for Python,
coverage$.$py, introduces a median overhead of 180% with the standard Python
interpreter. Slowdowns are even more extreme when using PyPy, a JIT-compiled
Python implementation, with coverage$.$py imposing a median overhead of 1,300%.
This performance degradation reduces the utility of coverage analysis in most
use cases, including testing and fuzzing, and precludes its use in deployment.
This paper presents SlipCover, a novel, near-zero overhead coverage analyzer
for Python. SlipCover works without modifications to either the Python
interpreter or PyPy. It first processes a program's AST to accurately identify
all branches and lines. SlipCover then dynamically rewrites Python bytecodes to
add lightweight instrumentation to each identified branch and line. At run
time, SlipCover periodically de-instruments already-covered lines and branches.
The result is extremely low overheads -- a median of just 5% -- making
SlipCover suitable for use in deployment. We show its efficiency can translate
to significant increases in the speed of coverage-based clients. As a proof of
concept, we integrate SlipCover into TPBT, a targeted property-based testing
system, and observe a 22x speedup.
