  The costs of training frontier AI models have grown dramatically in recent
years, but there is limited public data on the magnitude and growth of these
expenses. This paper develops a detailed cost model to address this gap,
estimating training costs using three approaches that account for hardware,
energy, cloud rental, and staff expenses. The analysis reveals that the
amortized cost to train the most compute-intensive models has grown
precipitously at a rate of 2.4x per year since 2016 (95% CI: 2.0x to 3.1x). For
key frontier models, such as GPT-4 and Gemini, the most significant expenses
are AI accelerator chips and staff costs, each costing tens of millions of
dollars. Other notable costs include server components (15-22%), cluster-level
interconnect (9-13%), and energy consumption (2-6%). If the trend of growing
development costs continues, the largest training runs will cost more than a
billion dollars by 2027, meaning that only the most well-funded organizations
will be able to finance frontier AI models.
