  Pre-training a transformer-based model for the language modeling task in a
large dataset and then fine-tuning it for downstream tasks has been found very
useful in recent years. One major advantage of such pre-trained language models
is that they can effectively absorb the context of each word in a sentence.
However, for tasks such as the answer selection task, the pre-trained language
models have not been extensively used yet. To investigate their effectiveness
in such tasks, in this paper, we adopt the pre-trained Bidirectional Encoder
Representations from Transformer (BERT) language model and fine-tune it on two
Question Answering (QA) datasets and three Community Question Answering (CQA)
datasets for the answer selection task. We find that fine-tuning the BERT model
for the answer selection task is very effective and observe a maximum
improvement of 13.1% in the QA datasets and 18.7% in the CQA datasets compared
to the previous state-of-the-art.
