  Conventional works generally employ a two-phase model in which a generator
selects the most important pieces, followed by a predictor that makes
predictions based on the selected pieces. However, such a two-phase model may
incur the degeneration problem where the predictor overfits to the noise
generated by a not yet well-trained generator and in turn, leads the generator
to converge to a sub-optimal model that tends to select senseless pieces. To
tackle this challenge, we propose Folded Rationalization (FR) that folds the
two phases of the rationale model into one from the perspective of text
semantic extraction. The key idea of FR is to employ a unified encoder between
the generator and predictor, based on which FR can facilitate a better
predictor by access to valuable information blocked by the generator in the
traditional two-phase model and thus bring a better generator. Empirically, we
show that FR improves the F1 score by up to 10.3% as compared to
state-of-the-art methods.
