  Rapidly increasing model scales coupled with steering methods such as
chain-of-thought prompting have led to drastic improvements in language model
reasoning. At the same time, models struggle with compositional generalization
and are far from human performance on many reasoning-based benchmarks.
Leveraging the success of chain-of-thought prompting, and also taking
inspiration from context-aware decoding (CAD), we explore input-based
contrasting methods to further encourage the type of reasoning induced by
chain-of-thought prompting. While work remains to stabilize these results
across datasets and models, the improvements we find warrant further
investigation into input-based steering methods for context-aware reasoning.
