  Cosmologists will soon be in a unique position. Observational noise will
gradually be replaced by cosmic variance as the dominant source of uncertainty
in an increasing number of observations. We reflect on the ramifications for
the discovery and verification of new models. If there are features in the full
data set that call for a new model, there will be no subsequent observations to
test that model's predictions. We give specific examples of the problem by
discussing the pitfalls of model discovery by prior adjustment in the context
of dark energy models and inflationary theories. We show how the gradual
release of data can mitigate this difficulty, allowing anomalies to be
identified, and new models to be proposed and tested. We advocate that
observers plan for the frugal release of data from future cosmic variance
limited observations.
