  The increasing complexity of modern very-large-scale integration (VLSI)
design highlights the significance of Electronic Design Automation (EDA)
technologies. Chip placement is a critical step in the EDA workflow, which
positions chip modules on the canvas with the goal of optimizing performance,
power, and area (PPA) metrics of final chip designs. Recent advances have
demonstrated the great potential of AI-based algorithms in enhancing chip
placement. However, due to the lengthy workflow of chip design, the evaluations
of these algorithms often focus on intermediate surrogate metrics, which are
easy to compute but frequently reveal a substantial misalignment with the
end-to-end performance (i.e., the final design PPA). To address this challenge,
we introduce ChiPBench, which can effectively facilitate research in chip
placement within the AI community. ChiPBench is a comprehensive benchmark
specifically designed to evaluate the effectiveness of existing AI-based chip
placement algorithms in improving final design PPA metrics. Specifically, we
have gathered 20 circuits from various domains (e.g., CPU, GPU, and
microcontrollers). These designs are compiled by executing the workflow from
the verilog source code, which preserves necessary physical implementation
kits, enabling evaluations for the placement algorithms on their impacts on the
final design PPA. We executed six state-of-the-art AI-based chip placement
algorithms on these designs and plugged the results of each single-point
algorithm into the physical implementation workflow to obtain the final PPA
results. Experimental results show that even if intermediate metric of a
single-point algorithm is dominant, while the final PPA results are
unsatisfactory. We believe that our benchmark will serve as an effective
evaluation framework to bridge the gap between academia and industry.
