  Executing machine learning inference tasks on resource-constrained edge
devices requires careful hardware-software co-design optimizations. Recent
examples have shown how transformer-based deep neural network models such as
ALBERT can be used to enable the execution of natural language processing (NLP)
inference on mobile systems-on-chip housing custom hardware accelerators.
However, while these existing solutions are effective in alleviating the
latency, energy, and area costs of running single NLP tasks, achieving
multi-task inference requires running computations over multiple variants of
the model parameters, which are tailored to each of the targeted tasks. This
approach leads to either prohibitive on-chip memory requirements or paying the
cost of off-chip memory access. This paper proposes adapter-ALBERT, an
efficient model optimization for maximal data reuse across different tasks. The
proposed model's performance and robustness to data compression methods are
evaluated across several language tasks from the GLUE benchmark. Additionally,
we demonstrate the advantage of mapping the model to a heterogeneous on-chip
memory architecture by performing simulations on a validated NLP edge
accelerator to extrapolate performance, power, and area improvements over the
execution of a traditional ALBERT model on the same hardware platform.
