  Differential privacy (DP) is increasingly used to protect the release of
hierarchical, tabular population data, such as census data. A common approach
for implementing DP in this setting is to release noisy responses to a
predefined set of queries. For example, this is the approach of the TopDown
algorithm used by the US Census Bureau. Such methods have an important
shortcoming: they cannot answer queries for which they were not optimized. An
appealing alternative is to generate DP synthetic data, which is drawn from
some generating distribution. Like the TopDown method, synthetic data can also
be optimized to answer specific queries, while also allowing the data user to
later submit arbitrary queries over the synthetic population data. To our
knowledge, there has not been a head-to-head empirical comparison of these
approaches. This study conducts such a comparison between the TopDown algorithm
and private synthetic data generation to determine how accuracy is affected by
query complexity, in-distribution vs. out-of-distribution queries, and privacy
guarantees. Our results show that for in-distribution queries, the TopDown
algorithm achieves significantly better privacy-fidelity tradeoffs than any of
the synthetic data methods we evaluated; for instance, in our experiments,
TopDown achieved at least $20\times$ lower error on counting queries than the
leading synthetic data method at the same privacy budget. Our findings suggest
guidelines for practitioners and the synthetic data research community.
