  Despite deep recurrent neural networks (RNNs) demonstrate strong performance
in text classification, training RNN models are often expensive and requires an
extensive collection of annotated data which may not be available. To overcome
the data limitation issue, existing approaches leverage either pre-trained word
embedding or sentence representation to lift the burden of training RNNs from
scratch. In this paper, we show that jointly learning sentence representations
from multiple text classification tasks and combining them with pre-trained
word-level and sentence level encoders result in robust sentence
representations that are useful for transfer learning. Extensive experiments
and analyses using a wide range of transfer and linguistic tasks endorse the
effectiveness of our approach.
