  GPUs have become the \emph{defacto} hardware devices for accelerating Deep
Neural Network (DNN) inference workloads. However, the conventional
\emph{sequential execution mode of DNN operators} in mainstream deep learning
frameworks cannot fully utilize GPU resources, even with the operator fusion
enabled, due to the increasing complexity of model structures and a greater
diversity of operators. Moreover, the \emph{inadequate operator launch order}
in parallelized execution scenarios can lead to GPU resource wastage and
unexpected performance interference among operators. In this paper, we propose
\emph{Opara}, a resource- and interference-aware DNN \underline{Op}erator
\underline{para}llel scheduling framework to accelerate DNN inference on GPUs.
Specifically, \emph{Opara} first employs \texttt{CUDA Streams} and \texttt{CUDA
Graph} to \emph{parallelize} the execution of multiple operators automatically.
To further expedite DNN inference, \emph{Opara} leverages the resource demands
of operators to judiciously adjust the operator launch order on GPUs,
overlapping the execution of compute-intensive and memory-intensive operators.
We implement and open source a prototype of \emph{Opara} based on PyTorch in a
\emph{non-intrusive} manner. Extensive prototype experiments with
representative DNN and Transformer-based models demonstrate that \emph{Opara}
outperforms the default sequential \texttt{CUDA Graph} in PyTorch and the
state-of-the-art operator parallelism systems by up to $1.68\times$ and
$1.29\times$, respectively, yet with acceptable runtime overhead.
