  We review the cost of training large-scale language models, and the drivers
of these costs. The intended audience includes engineers and scientists
budgeting their model-training experiments, as well as non-practitioners trying
to make sense of the economics of modern-day Natural Language Processing (NLP).
