  We introduce ComplexTempQA,a large-scale dataset consisting of over 100
million question-answer pairs designed to tackle the challenges in temporal
question answering. ComplexTempQA significantly surpasses existing benchmarks
like HOTPOTQA, TORQUE, and TEQUILA in scale and scope. Utilizing data from
Wikipedia and Wikidata, the dataset covers questions spanning over two decades
and offers an unmatched breadth of topics. We introduce a unique taxonomy that
categorizes questions as attributes, comparisons, and counting questions, each
revolving around events, entities, and time periods. One standout feature of
ComplexTempQA is the high complexity of its questions, which demand effective
capabilities for answering such as across-time comparison, temporal
aggregation, and multi-hop reasoning involving temporal event ordering and
entity recognition. Additionally, each question is accompanied by detailed
metadata, including specific time scopes, allowing for comprehensive evaluation
and enhancement of the temporal reasoning abilities of large language models.
ComplexTempQA serves both as a testing ground for developing sophisticated AI
models and as a foundation for advancing research in question answering,
information retrieval, and language understanding. Dataset and code are freely
available at: https://github.com/DataScienceUIBK/ComplexTempQA.
