  The progress of humanity is driven by those successful discoveries
accompanied by countless failed experiments. Researchers often seek the
potential research directions by reading and then verifying them through
experiments. The process imposes a significant burden on researchers. In the
past decade, the data-driven black-box deep learning method has demonstrated
its effectiveness in a wide range of real-world scenarios, which exacerbates
the experimental burden of researchers and thus renders the potential
successful discoveries veiled. Therefore, automating such a research and
development (R&D) process is an urgent need. In this paper, we serve as the
first effort to formalize the goal by proposing a Real-world Data-centric
automatic R&D Benchmark, namely RD2Bench. RD2Bench benchmarks all the
operations in data-centric automatic R&D (D-CARD) as a whole to navigate future
work toward our goal directly. We focus on evaluating the interaction and
synergistic effects of various model capabilities and aiding in selecting
well-performing trustworthy models. Although RD2Bench is very challenging to
the state-of-the-art (SOTA) large language model (LLM) named GPT-4, indicating
ample research opportunities and more research efforts, LLMs possess promising
potential to bring more significant development to D-CARD: They are able to
implement some simple methods without adopting any additional techniques. We
appeal to future work to take developing techniques for tackling automatic R&D
into consideration, thus bringing the opportunities of the potential
revolutionary upgrade to human productivity.
