  We study randomized experiments in a service system when stochastic
congestion can arise from temporarily limited supply and/or demand. Such
congestion gives rise to cross-unit interference between the waiting customers,
and analytic strategies that do not account for this interference may be
biased. In current practice, one of the most widely used ways to address
stochastic congestion is to use switchback experiments that alternatively turn
a target intervention on and off for the whole system. We find, however, that
under a queueing model for stochastic congestion, the standard way of analyzing
switchbacks is inefficient, and that estimators that leverage the queueing
model can be materially more accurate. We also consider a new class of
experimental design, which can be used to estimate a policy gradient of the
dynamic system using only unit-level randomization, thus alleviating key
practical challenges that arise in running a switchback.
