  We present a comprehensive benchmark dataset for Knowledge Graph Question
Answering in Materials Science (KGQA4MAT), with a focus on metal-organic
frameworks (MOFs). A knowledge graph for metal-organic frameworks (MOF-KG) has
been constructed by integrating structured databases and knowledge extracted
from the literature. To enhance MOF-KG accessibility for domain experts, we aim
to develop a natural language interface for querying the knowledge graph. We
have developed a benchmark comprised of 161 complex questions involving
comparison, aggregation, and complicated graph structures. Each question is
rephrased in three additional variations, resulting in 644 questions and 161 KG
queries. To evaluate the benchmark, we have developed a systematic approach for
utilizing the LLM, ChatGPT, to translate natural language questions into formal
KG queries. We also apply the approach to the well-known QALD-9 dataset,
demonstrating ChatGPT's potential in addressing KGQA issues for different
platforms and query languages. The benchmark and the proposed approach aim to
stimulate further research and development of user-friendly and efficient
interfaces for querying domain-specific materials science knowledge graphs,
thereby accelerating the discovery of novel materials.
