  We investigate response selection for multi-turn conversation in
retrieval-based chatbots. Existing studies pay more attention to the matching
between utterances and responses by calculating the matching score based on
learned features, leading to insufficient model reasoning ability. In this
paper, we propose a graph-reasoning network (GRN) to address the problem. GRN
first conducts pre-training based on ALBERT using next utterance prediction and
utterance order prediction tasks specifically devised for response selection.
These two customized pre-training tasks can endow our model with the ability of
capturing semantical and chronological dependency between utterances. We then
fine-tune the model on an integrated network with sequence reasoning and graph
reasoning structures. The sequence reasoning module conducts inference based on
the highly summarized context vector of utterance-response pairs from the
global perspective. The graph reasoning module conducts the reasoning on the
utterance-level graph neural network from the local perspective. Experiments on
two conversational reasoning datasets show that our model can dramatically
outperform the strong baseline methods and can achieve performance which is
close to human-level.
