  Texture editing is a crucial task in 3D modeling that allows users to
automatically manipulate the surface materials of 3D models. However, the
inherent complexity of 3D models and the ambiguous text description lead to the
challenge in this task. To address this challenge, we propose ITEM3D, a
\textbf{T}exture \textbf{E}diting \textbf{M}odel designed for automatic
\textbf{3D} object editing according to the text \textbf{I}nstructions.
Leveraging the diffusion models and the differentiable rendering, ITEM3D takes
the rendered images as the bridge of text and 3D representation, and further
optimizes the disentangled texture and environment map. Previous methods
adopted the absolute editing direction namely score distillation sampling (SDS)
as the optimization objective, which unfortunately results in the noisy
appearance and text inconsistency. To solve the problem caused by the ambiguous
text, we introduce a relative editing direction, an optimization objective
defined by the noise difference between the source and target texts, to release
the semantic ambiguity between the texts and images. Additionally, we gradually
adjust the direction during optimization to further address the unexpected
deviation in the texture domain. Qualitative and quantitative experiments show
that our ITEM3D outperforms the state-of-the-art methods on various 3D objects.
We also perform text-guided relighting to show explicit control over lighting.
Our project page: https://shengqiliu1.github.io/ITEM3D.
