  Real-world applications of neural language models often involve running many
different models over the same corpus. The high computational cost of these
runs has led to interest in techniques that can reuse the contextualized
embeddings produced in previous runs to speed training and inference of future
ones. We refer to this approach as embedding recycling (ER). While multiple ER
techniques have been proposed, their practical effectiveness is still unknown
because existing evaluations consider very few models and do not adequately
account for overhead costs. We perform an extensive evaluation of ER across
eight different models (17 to 900 million parameters) and fourteen tasks in
English. We show how a simple ER technique that caches activations from an
intermediate layer of a pretrained model, and learns task-specific adapters on
the later layers, is broadly effective. For the best-performing baseline in our
experiments (DeBERTa-v2 XL), adding a precomputed cache results in a >90%
speedup during training and 87-91% speedup for inference, with negligible
impact on accuracy. Our analysis reveals important areas of future work.
