  In this paper, we introduce Dreamweaver, which belongs to a new class of
auto-regressive decision-making models known as large reasoning models (LRMs).
Dreamweaver is designed to improve 3D floorplanning in electronic design
automation (EDA) via an architecture that melds advancements in
sequence-to-sequence reinforcement learning algorithms. A significant advantage
of our approach is its ability to effectively reason over large discrete action
spaces, which is essential for handling the numerous potential positions for
various functional blocks in floorplanning. Additionally, Dreamweaver
demonstrates strong performance even when trained on entirely random
trajectories, showcasing its capacity to leverage sub-optimal or non-expert
trajectories to enhance its results. This innovative approach contributes to
streamlining the integrated circuit (IC) design flow and reducing the high
computational costs typically associated with floorplanning. We evaluate its
performance against a current state-of-the-art method, highlighting notable
improvements.
