  Weakly-supervised table question-answering(TableQA) models have achieved
state-of-art performance by using pre-trained BERT transformer to jointly
encoding a question and a table to produce structured query for the question.
However, in practical settings TableQA systems are deployed over table corpora
having topic and word distributions quite distinct from BERT's pretraining
corpus. In this work we simulate the practical topic shift scenario by
designing novel challenge benchmarks WikiSQL-TS and WikiTQ-TS, consisting of
train-dev-test splits in five distinct topic groups, based on the popular
WikiSQL and WikiTableQuestions datasets. We empirically show that, despite
pre-training on large open-domain text, performance of models degrades
significantly when they are evaluated on unseen topics. In response, we propose
T3QA (Topic Transferable Table Question Answering) a pragmatic adaptation
framework for TableQA comprising of: (1) topic-specific vocabulary injection
into BERT, (2) a novel text-to-text transformer generator (such as T5, GPT2)
based natural language question generation pipeline focused on generating topic
specific training data, and (3) a logical form reranker. We show that T3QA
provides a reasonably good baseline for our topic shift benchmarks. We believe
our topic split benchmarks will lead to robust TableQA solutions that are
better suited for practical deployment.
