  Information Extraction is a well-researched area of Natural Language
Processing with applications in web search and question answering concerned
with identifying entities and relationships between them as expressed in a
given context, usually a sentence of a paragraph of running text. Given the
importance of the task, several datasets and benchmarks have been curated over
the years. However, focusing on running text alone leaves out tables which are
common in many structured documents and in which pairs of entities also
co-occur in context (e.g., the same row of the table). While there are recent
papers on relation extraction from tables in the literature, their experimental
evaluations have been on ad-hoc datasets for the lack of a standard benchmark.
This paper helps close that gap. We introduce an annotation framework and a
dataset of 217,834 tables from Wikipedia which are annotated with 28 relations,
using both classifiers and carefully designed queries over a reference
knowledge graph. Binary classifiers are then applied to the resulting dataset
to remove false positives, resulting in an average annotation accuracy of 94%.
The resulting dataset is the first of its kind to be made publicly available.
