  Multilingual sentence representations pose a great advantage for low-resource
languages that do not have enough data to build monolingual models on their
own. These multilingual sentence representations have been separately exploited
by few research for document and sentence alignment. However, most of the
low-resource languages are under-represented in these pre-trained models. Thus,
in the context of low-resource languages, these models have to be fine-tuned
for the task at hand, using additional data sources. This paper presents a
weighting mechanism that makes use of available small-scale parallel corpora to
improve the performance of multilingual sentence representations on document
and sentence alignment. Experiments are conducted with respect to two
low-resource languages, Sinhala and Tamil. Results on a newly created dataset
of Sinhala-English, Tamil-English, and Sinhala-Tamil show that this new
weighting mechanism significantly improves both document and sentence
alignment. This dataset, as well as the source-code, is publicly released.
