  A growing body of research has been dedicated to DL model testing. However,
there is still limited work on testing DL libraries, which serve as the
foundations for building, training, and running DL models. Prior work on
fuzzing DL libraries can only generate tests for APIs which have been invoked
by documentation examples, developer tests, or DL models, leaving a large
number of APIs untested. In this paper, we propose DeepREL, the first approach
to automatically inferring relational APIs for more effective DL library
fuzzing. Our basic hypothesis is that for a DL library under test, there may
exist a number of APIs sharing similar input parameters and outputs; in this
way, we can easily "borrow" test inputs from invoked APIs to test other
relational APIs. Furthermore, we formalize the notion of value equivalence and
status equivalence for relational APIs to serve as the oracle for effective bug
finding. We have implemented DeepREL as a fully automated end-to-end relational
API inference and fuzzing technique for DL libraries, which 1) automatically
infers potential API relations based on API syntactic or semantic information,
2) synthesizes concrete test programs for invoking relational APIs, 3)
validates the inferred relational APIs via representative test inputs, and
finally 4) performs fuzzing on the verified relational APIs to find potential
inconsistencies. Our evaluation on two of the most popular DL libraries,
PyTorch and TensorFlow, demonstrates that DeepREL can cover 157% more APIs than
state-of-the-art FreeFuzz. To date, DeepREL has detected 162 bugs in total,
with 106 already confirmed by the developers as previously unknown bugs.
Surprisingly, DeepREL has detected 13.5% of the high-priority bugs for the
entire PyTorch issue-tracking system in a three-month period. Also, besides the
162 code bugs, we have also detected 14 documentation bugs (all confirmed).
