  The emergence of foundation models in Computer Vision and Natural Language
Processing have resulted in immense progress on downstream tasks. This progress
was enabled by datasets with billions of training examples. Similar benefits
are yet to be unlocked for quantum chemistry, where the potential of deep
learning is constrained by comparatively small datasets with 100k to 20M
training examples. These datasets are limited in size because the labels are
computed using the accurate (but computationally demanding) predictions of
Density Functional Theory (DFT). Notably, prior DFT datasets were created using
CPU supercomputers without leveraging hardware acceleration. In this paper, we
take a first step towards utilising hardware accelerators by introducing the
data generator PySCF$_{\text{IPU}}$ using Intelligence Processing Units (IPUs).
This allowed us to create the dataset QM1B with one billion training examples
containing 9-11 heavy atoms. We demonstrate that a simple baseline neural
network (SchNet 9M) improves its performance by simply increasing the amount of
training data without additional inductive biases. To encourage future
researchers to use QM1B responsibly, we highlight several limitations of QM1B
and emphasise the low-resolution of our DFT options, which also serves as
motivation for even larger, more accurate datasets. Code and dataset are
available on Github: http://github.com/graphcore-research/pyscf-ipu
