  A good open-domain chatbot should avoid presenting contradictory responses
about facts or opinions in a conversational session, known as its consistency
capacity. However, evaluating the consistency capacity of a chatbot is still
challenging. Employing human judges to interact with chatbots on purpose to
check their capacities is costly and low-efficient, and difficult to get rid of
subjective bias. In this paper, we propose the Addressing Inquiries about
History (AIH), an efficient and practical framework for the consistency
evaluation. At the conversation stage, AIH attempts to address appropriate
inquiries about the dialogue history to induce the chatbot to redeclare the
historical facts or opinions. We carry out the conversation between chatbots,
which is more efficient than the human-bot interaction and can also alleviate
the subjective bias. In this way, we manage to rapidly obtain a dialog session
that contains responses with high contradiction possibilities. At the
contradiction recognition stage, we can either employ human judges or a natural
language inference (NLI) model to recognize whether the answers to the
inquiries are contradictory with history. Finally, we are able to rank chatbots
according to the contradiction statistics. Experiments on open-domain chatbots
show that our approach can efficiently and reliably assess the consistency
capacity of chatbots and achieve a high ranking correlation with the human
evaluation. We release the framework and hope to help improve the consistency
capacity of chatbots. \footnote{\url{https://github.com/ictnlp/AIH}}
