  Abstractive summarization systems today produce fluent and relevant output,
but often "hallucinate" statements not supported by the source text. We analyze
the connection between hallucinations and training data, and find evidence that
models hallucinate because they train on target summaries that are unsupported
by the source. Based on our findings, we present PINOCCHIO, a new decoding
method that improves the consistency of a transformer-based abstractive
summarizer by constraining beam search to avoid hallucinations. Given the model
states and outputs at a given step, PINOCCHIO detects likely model
hallucinations based on various measures of attribution to the source text.
PINOCCHIO backtracks to find more consistent output, and can opt to produce no
summary at all when no consistent generation can be found. In experiments, we
find that PINOCCHIO improves the consistency of generation (in terms of F1) by
an average of~67% on two abstractive summarization datasets.
