  Knowledge distillation is a form of model compression that allows artificial
neural networks of different sizes to learn from one another. Its main
application is the compactification of large deep neural networks to free up
computational resources, in particular on edge devices. In this article, we
consider proton-proton collisions at the High-Luminosity LHC (HL-LHC) and
demonstrate a successful knowledge transfer from an event-level graph neural
network (GNN) to a particle-level small deep neural network (DNN). Our
algorithm, DistillNet, is a DNN that is trained to learn about the provenance
of particles, as provided by the soft labels that are the GNN outputs, to
predict whether or not a particle originates from the primary interaction
vertex. The results indicate that for this problem, which is one of the main
challenges at the HL-LHC, there is minimal loss during the transfer of
knowledge to the small student network, while improving significantly the
computational resource needs compared to the teacher. This is demonstrated for
the distilled student network on a CPU, as well as for a quantized and pruned
student network deployed on a field-programmable gate array. Our study proves
that knowledge transfer between networks of different complexity can be used
for fast artificial intelligence (AI) in high-energy physics that improves the
expressiveness of observables over non-AI-based reconstruction algorithms. Such
an approach can become essential at the HL-LHC experiments, e.g., to comply
with the resource budget of their trigger stages.
