  Quality of questions and answers from community support websites (e.g.
Microsoft Developers Network, Stackoverflow, Github, etc.) is difficult to
define and a prediction model of quality questions and answers is even more
challenging to implement. Previous works have addressed the question quality
models and answer quality models separately using meta-features like number of
up-votes, trustworthiness of the person posting the questions or answers,
titles of the post, and context naive natural language processing features.
However, there is a lack of an integrated question-answer quality model for
community question answering websites in the literature. In this brief paper,
we tackle the quality Q&A modeling problems from the community support websites
using a recently developed deep learning model using bidirectional
transformers. We investigate the applicability of transfer learning on Q&A
quality modeling using Bidirectional Encoder Representations from Transformers
(BERT) trained on a separate tasks originally using Wikipedia. It is found that
a further pre-training of BERT model along with finetuning on the Q&As
extracted from Microsoft Developer Network (MSDN) can boost the performance of
automated quality prediction to more than 80%. Furthermore, the implementations
are carried out for deploying the finetuned model in real-time scenario using
AzureML in Azure knowledge base system.
