  Empirically-determined scaling laws have been broadly successful in
predicting the evolution of large machine learning models with training data
and number of parameters. As a consequence, they have been useful for
optimizing the allocation of limited resources, most notably compute time.
  In certain applications, storage space is an important constraint, and data
format needs to be chosen carefully as a consequence. Computer vision is a
prominent example: images are inherently analog, but are always stored in a
digital format using a finite number of bits. Given a dataset of digital
images, the number of bits $L$ to store each of them can be further reduced
using lossy data compression. This, however, can degrade the quality of the
model trained on such images, since each example has lower resolution.
  In order to capture this trade-off and optimize storage of training data, we
propose a `storage scaling law' that describes the joint evolution of test
error with sample size and number of bits per image. We prove that this law
holds within a stylized model for image compression, and verify it empirically
on two computer vision tasks, extracting the relevant parameters. We then show
that this law can be used to optimize the lossy compression level. At given
storage, models trained on optimally compressed images present a significantly
smaller test error with respect to models trained on the original data.
Finally, we investigate the potential benefits of randomizing the compression
level.
