  We show that supervised neural information retrieval (IR) models are prone to
learning sparse attention patterns over passage tokens, which can result in key
phrases including named entities receiving low attention weights, eventually
leading to model under-performance. Using a novel targeted synthetic data
generation method that identifies poorly attended entities and conditions the
generation episodes on those, we teach neural IR to attend more uniformly and
robustly to all entities in a given passage. On two public IR benchmarks, we
empirically show that the proposed method helps improve both the model's
attention patterns and retrieval performance, including in zero-shot settings.
