  Image-guided object assembly represents a burgeoning research topic in
computer vision. This paper introduces a novel task: translating multi-view
images of a structural 3D model (for example, one constructed with building
blocks drawn from a 3D-object library) into a detailed sequence of assembly
instructions executable by a robotic arm. Fed with multi-view images of the
target 3D model for replication, the model designed for this task must address
several sub-tasks, including recognizing individual components used in
constructing the 3D model, estimating the geometric pose of each component, and
deducing a feasible assembly order adhering to physical rules. Establishing
accurate 2D-3D correspondence between multi-view images and 3D objects is
technically challenging. To tackle this, we propose an end-to-end model known
as the Neural Assembler. This model learns an object graph where each vertex
represents recognized components from the images, and the edges specify the
topology of the 3D model, enabling the derivation of an assembly plan. We
establish benchmarks for this task and conduct comprehensive empirical
evaluations of Neural Assembler and alternative solutions. Our experiments
clearly demonstrate the superiority of Neural Assembler.
