  For simple classification tasks, we show that users can benefit from the
advantages of using small, local, generative language models instead of large
commercial models without a trade-off in performance or introducing extra
labelling costs. These advantages, including those around privacy,
availability, cost, and explainability, are important both in commercial
applications and in the broader democratisation of AI. Through experiments on
17 sentence classification tasks (2-4 classes), we show that penalised logistic
regression on the embeddings from a small LLM equals (and usually betters) the
performance of a large LLM in the "tens-of-shot" regime. This requires no more
labelled instances than are needed to validate the performance of the large
LLM. Finally, we extract stable and sensible explanations for classification
decisions.
