  This work proposes a challenging yet more realistic setting for zero-shot
cross-task generalization: zero-shot instruction following, presuming the
existence of a paragraph-style task definition while no demonstrations exist.
To better learn the task supervision from the definition, we propose two
strategies: first, to automatically find out the critical sentences in the
definition; second, a ranking objective to force the model to generate the gold
outputs with higher probabilities when those critical parts are highlighted in
the definition. The joint efforts of the two strategies yield state-of-the-art
performance on the Super-NaturalInstructions. Our code is available on GitHub.
