  Large language models (LLMs) tend to inadequately integrate input context
during text generation, relying excessively on encoded prior knowledge in model
parameters, potentially resulting in generated text with factual
inconsistencies or contextually unfaithful content. LLMs utilize two primary
knowledge sources: 1) prior (parametric) knowledge from pretraining, and 2)
contextual (non-parametric) knowledge from input prompts. The study addresses
the open question of how LLMs effectively balance these knowledge sources
during the generation process, specifically in the context of open-domain
question answering. To address this issue, we introduce a novel approach
integrating contrastive decoding with adversarial irrelevant passages as
negative samples to enhance robust context grounding during generation.
Notably, our method operates at inference time without requiring further
training. We conduct comprehensive experiments to demonstrate its applicability
and effectiveness, providing empirical evidence showcasing its superiority over
existing methodologies. Our code is publicly available at:
https://github.com/amazon-science/ContextualUnderstanding-ContrastiveDecoding.
