  Machine learning systems require representations of the real world for
training and testing - they require data, and lots of it. Collecting data at
scale has logistical and ethical challenges, and synthetic data promises a
solution to these challenges. Instead of needing to collect photos of real
people's faces to train a facial recognition system, a model creator could
create and use photo-realistic, synthetic faces. The comparative ease of
generating this synthetic data rather than relying on collecting data has made
it a common practice. We present two key risks of using synthetic data in model
development. First, we detail the high risk of false confidence when using
synthetic data to increase dataset diversity and representation. We base this
in the examination of a real world use-case of synthetic data, where synthetic
datasets were generated for an evaluation of facial recognition technology.
Second, we examine how using synthetic data risks circumventing consent for
data usage. We illustrate this by considering the importance of consent to the
U.S. Federal Trade Commission's regulation of data collection and affected
models. Finally, we discuss how these two risks exemplify how synthetic data
complicates existing governance and ethical practice; by decoupling data from
those it impacts, synthetic data is prone to consolidating power away those
most impacted by algorithmically-mediated harm.
