  When pretrained language models (LMs) are applied to discriminative tasks
such as multiple-choice questions, they place probability mass on vocabulary
tokens that aren't among the given answer choices. Spreading probability mass
across multiple surface forms with identical meaning (such as "bath" and
"bathtub") is thought to cause an underestimation of a model's true
performance, referred to as the "surface form competition" (SFC) hypothesis.
This has motivated the introduction of various probability normalization
methods. However, many core questions remain unanswered. How do we measure SFC?
Are there direct ways of reducing it, and does doing so improve task
performance?
  We propose a mathematical formalism for SFC which allows us to quantify and
bound its impact for the first time. We identify a simple method for reducing
it -- namely, increasing probability mass on the given answer choices by a)
including them in the prompt and b) using in-context learning with even just
one example. We show this method eliminates the impact of SFC in the majority
of instances. Our experiments on three diverse datasets and six LMs reveal
several additional surprising findings. For example, both normalization and
prompting methods for reducing SFC can be ineffective or even detrimental to
task performance for some LMs. We conclude with practical insights for
effectively prompting LMs for multiple-choice tasks.
