  While Physics-Based Simulation (PBS) can accurately drape a 3D garment on a
3D body, it remains too costly for real-time applications, such as virtual
try-on. By contrast, inference in a deep network, requiring a single forward
pass, is much faster. Taking advantage of this, we propose a novel architecture
to fit a 3D garment template to a 3D body. Specifically, we build upon the
recent progress in 3D point cloud processing with deep networks to extract
garment features at varying levels of detail, including point-wise, patch-wise
and global features. We fuse these features with those extracted in parallel
from the 3D body, so as to model the cloth-body interactions. The resulting
two-stream architecture, which we call as GarNet, is trained using a loss
function inspired by physics-based modeling, and delivers visually plausible
garment shapes whose 3D points are, on average, less than 1 cm away from those
of a PBS method, while running 100 times faster. Moreover, the proposed method
can model various garment types with different cutting patterns when parameters
of those patterns are given as input to the network.
