  Two observational methods are currently being used to monitor post-deployment
vaccine effectiveness: the obvious crude method comparing rate testing positive
per head of vaccinated population with that rate per head of unvaccinated
population; and the test-negative case control (TNCC) method. The two methods
give very different results. We want to know whether either method is reliable.
  We assume either a homogeneous population or one partitioned into two
homogeneous subsets which differ only in their not-directly-observable
healthcare-seeking behaviour including probability of getting vaccinated. We
first consider uniform independent priors on the probabilities of being
hospitalised conditional on subset, vaccination status, and infection status.
We simulate from the resulting model and observe the TNCC estimate, the crude
estimate, and the Bayesian central 95% confidence interval on vaccine
effectiveness represented as log ratio of odds ratios for infection with and
without vaccination.
  With these wide open priors, even when the population is homogeneous, the
Bayesian 95% confidence interval typically has a width of nearly 4 nats
(55-fold), implying too much uncertainty for the data collected to be of any
use in monitoring effectiveness. There do exist some tight priors under which
the data is useful: some lead to TNCC being more accurate while with others the
crude estimate is more accurate.
  Thus using only data from those spontaneously choosing to be tested, we find
that neither method is reliably better than the other, and indeed that the
desired information is not present in this data. We conclude that effective
monitoring of vaccine effectiveness and side-effects requires either strong
information on the population's behaviour, or ongoing randomised controlled
trials (RCTs), rather than just choosing whichever of TNCC and crude estimate
gives the result we prefer to find.
