  Methods for learning sentence representations have been actively developed in
recent years. However, the lack of pre-trained models and datasets annotated at
the sentence level has been a problem for low-resource languages such as Polish
which led to less interest in applying these methods to language-specific
tasks. In this study, we introduce two new Polish datasets for evaluating
sentence embeddings and provide a comprehensive evaluation of eight sentence
representation methods including Polish and multilingual models. We consider
classic word embedding models, recently developed contextual embeddings and
multilingual sentence encoders, showing strengths and weaknesses of specific
approaches. We also examine different methods of aggregating word vectors into
a single sentence vector.
