  Despite significant progress in understanding and improving faithfulness in
abstractive summarization, the question of how decoding strategies affect
faithfulness is less studied. We present a systematic study of the effect of
generation techniques such as beam search and nucleus sampling on faithfulness
in abstractive summarization. We find a consistent trend where beam search with
large beam sizes produces the most faithful summaries while nucleus sampling
generates the least faithful ones. We propose two faithfulness-aware generation
methods to further improve faithfulness over current generation techniques: (1)
ranking candidates generated by beam search using automatic faithfulness
metrics and (2) incorporating lookahead heuristics that produce a faithfulness
score on the future summary. We show that both generation methods significantly
improve faithfulness across two datasets as evaluated by four automatic
faithfulness metrics and human evaluation. To reduce computational cost, we
demonstrate a simple distillation approach that allows the model to generate
faithful summaries with just greedy decoding. Our code is publicly available at
https://github.com/amazon-science/faithful-summarization-generation
