  As the COVID-19 pandemic progresses, researchers are reporting findings of
randomized trials comparing standard care with care augmented by experimental
drugs. The trials have small sample sizes, so estimates of treatment effects
are imprecise. Seeing imprecision, clinicians reading research articles may
find it difficult to decide when to treat patients with experimental drugs.
Whatever decision criterion one uses, there is always some probability that
random variation in trial outcomes will lead to prescribing sub-optimal
treatments. A conventional practice when comparing standard care and an
innovation is to choose the innovation only if the estimated treatment effect
is positive and statistically significant. This practice defers to standard
care as the status quo. To evaluate decision criteria, we use the concept of
near-optimality, which jointly considers the probability and magnitude of
decision errors. An appealing decision criterion from this perspective is the
empirical success rule, which chooses the treatment with the highest observed
average patient outcome in the trial. Considering the design of recent and
ongoing COVID-19 trials, we show that the empirical success rule yields
treatment results that are much closer to optimal than those generated by
prevailing decision criteria based on hypothesis tests.
