  A riddle is a question or statement with double or veiled meanings, followed
by an unexpected answer. Solving riddle is a challenging task for both machine
and human, testing the capability of understanding figurative, creative natural
language and reasoning with commonsense knowledge. We introduce BiRdQA, a
bilingual multiple-choice question answering dataset with 6614 English riddles
and 8751 Chinese riddles. For each riddle-answer pair, we provide four
distractors with additional information from Wikipedia. The distractors are
automatically generated at scale with minimal bias. Existing monolingual and
multilingual QA models fail to perform well on our dataset, indicating that
there is a long way to go before machine can beat human on solving tricky
riddles. The dataset has been released to the community.
