  Static and dynamic computational graphs represent two distinct approaches to
constructing deep learning frameworks. The former prioritizes compiler-based
optimizations, while the latter focuses on programmability and
user-friendliness. The recent release of PyTorch 2.0, which supports compiling
arbitrary deep learning programs in Python, signifies a new direction in the
evolution of deep learning infrastructure to incorporate compiler techniques in
a more dynamic manner and support more dynamic language features like dynamic
control flows and closures. Given PyTorch's seamless integration with Python,
its compiler aims to support arbitrary deep learning code written in Python.
However, the inherent dynamism of Python poses challenges to the completeness
and robustness of the compiler. While recent research has introduced fuzzing to
test deep learning compilers, there is still a lack of comprehensive analysis
on how to test dynamic features. To address this issue, we propose several code
transformations to generate test cases involving dynamic features. These
transformations preserve the program's semantics, ensuring that any discrepancy
between the transformed and original programs indicates the presence of a bug.
Through our approach, we have successfully identified twenty previously unknown
bugs in the PyTorch compiler and its underlying tensor compiler Triton.
