  Large Language Models (LLMs) have democratized synthetic data generation,
which in turn has the potential to simplify and broaden a wide gamut of NLP
tasks. Here, we tackle a pervasive problem in synthetic data generation: its
generative distribution often differs from the distribution of real-world data
researchers care about (in other words, it is unfaithful). In a case study on
sarcasm detection, we study three strategies to increase the faithfulness of
synthetic data: grounding, filtering, and taxonomy-based generation. We
evaluate these strategies using the performance of classifiers trained with
generated synthetic data on real-world data. While all three strategies improve
the performance of classifiers, we find that grounding works best for the task
at hand. As synthetic data generation plays an ever-increasing role in NLP
research, we expect this work to be a stepping stone in improving its utility.
We conclude this paper with some recommendations on how to generate
high(er)-fidelity synthetic data for specific tasks.
