  Inspired by traditional handmade crafts, where a person improvises assemblies
based on the available objects, we formally introduce the Craft Assembly Task.
It is a robotic assembly task that involves building an accurate representation
of a given target object using the available objects, which do not directly
correspond to its parts. In this work, we focus on selecting the subset of
available objects for the final craft, when the given input is an RGB image of
the target in the wild. We use a mask segmentation neural network to identify
visible parts, followed by retrieving labelled template meshes. These meshes
undergo pose optimization to determine the most suitable template. Then, we
propose to simplify the parts of the transformed template mesh to primitive
shapes like cuboids or cylinders. Finally, we design a search algorithm to find
correspondences in the scene based on local and global proportions. We develop
baselines for comparison that consider all possible combinations, and choose
the highest scoring combination for common metrics used in foreground maps and
mask accuracy. Our approach achieves comparable results to the baselines for
two different scenes, and we show qualitative results for an implementation in
a real-world scenario.
