  Creating high-quality materials in computer graphics is a challenging and
time-consuming task, which requires great expertise. To simplify this process,
we introduce MatFuse, a unified approach that harnesses the generative power of
diffusion models for creation and editing of 3D materials. Our method
integrates multiple sources of conditioning, including color palettes,
sketches, text, and pictures, enhancing creative possibilities and granting
fine-grained control over material synthesis. Additionally, MatFuse enables
map-level material editing capabilities through latent manipulation by means of
a multi-encoder compression model which learns a disentangled latent
representation for each map. We demonstrate the effectiveness of MatFuse under
multiple conditioning settings and explore the potential of material editing.
Finally, we assess the quality of the generated materials both quantitatively
in terms of CLIP-IQA and FID scores and qualitatively by conducting a user
study. Source code for training MatFuse and supplemental materials are publicly
available at https://gvecchio.com/matfuse.
