  Frequently Asked Question (FAQ) retrieval is an important task where the
objective is to retrieve an appropriate Question-Answer (QA) pair from a
database based on a user's query. We propose a FAQ retrieval system that
considers the similarity between a user's query and a question as well as the
relevance between the query and an answer. Although a common approach to FAQ
retrieval is to construct labeled data for training, it takes annotation costs.
Therefore, we use a traditional unsupervised information retrieval system to
calculate the similarity between the query and question. On the other hand, the
relevance between the query and answer can be learned by using QA pairs in a
FAQ database. The recently-proposed BERT model is used for the relevance
calculation. Since the number of QA pairs in FAQ page is not enough to train a
model, we cope with this issue by leveraging FAQ sets that are similar to the
one in question. We evaluate our approach on two datasets. The first one is
localgovFAQ, a dataset we construct in a Japanese administrative municipality
domain. The second is StackExchange dataset, which is the public dataset in
English. We demonstrate that our proposed method outperforms baseline methods
on these datasets.
