  Graph Neural Network (GNN) inference is used in many real-world applications.
Data sparsity in GNN inference, including sparsity in the input graph and the
GNN model, offer opportunities to further speed up inference. Also, many
pruning techniques have been proposed for model compression that increase the
data sparsity of GNNs. We propose Dynasparse, a comprehensive hardware-software
codesign on FPGA to accelerate GNN inference through dynamic sparsity
exploitation. For this, we decouple the GNN computation kernels from the basic
computation primitives, and explore hardware-software codesign as follows: 1)
Hardware design: We propose a novel unified accelerator design on FPGA to
efficiently execute various computation primitives. We develop a customized
soft processor that is tightly coupled with the accelerator to execute a
runtime system. Moreover, we develop efficient hardware mechanisms to profile
the data sparsity and perform on-the-fly data format transformation to prepare
the input data for various computation primitives; 2) Software design: We
develop a runtime system that works synergistically with the accelerator to
perform dynamic kernel-to-primitive mapping based on data sparsity. We
implement Dynasparse on a state-of-the-art FPGA platform, Xilinx Alveo U250,
and evaluate the design using widely used GNN models (GCN, GraphSAGE, GIN and
SGC). For the above GNN models and various input graphs, the proposed
accelerator and dynamic kernel-to-primitive mapping reduces the inference
latency by $3.73\times$ on the average compared with the static mapping
strategies employed in the state-of-the-art GNN accelerators. Compared with
state-of-the-art CPU (GPU) implementations, Dynasparse achieves up to
$56.9\times$ ($2.37\times$) speedup in end-to-end latency.
