  Manipulating visual attributes of images through human-written text is a very
challenging task. On the one hand, models have to learn the manipulation
without the ground truth of the desired output. On the other hand, models have
to deal with the inherent ambiguity of natural language. Previous research
usually requires either the user to describe all the characteristics of the
desired image or to use richly-annotated image captioning datasets. In this
work, we propose a novel unsupervised approach, based on image-to-image
translation, that alters the attributes of a given image through a command-like
sentence such as "change the hair color to black". Contrarily to
state-of-the-art approaches, our model does not require a human-annotated
dataset nor a textual description of all the attributes of the desired image,
but only those that have to be modified. Our proposed model disentangles the
image content from the visual attributes, and it learns to modify the latter
using the textual description, before generating a new image from the content
and the modified attribute representation. Because text might be inherently
ambiguous (blond hair may refer to different shadows of blond, e.g. golden,
icy, sandy), our method generates multiple stochastic versions of the same
translation. Experiments show that the proposed model achieves promising
performances on two large-scale public datasets: CelebA and CUB. We believe our
approach will pave the way to new avenues of research combining textual and
speech commands with visual attributes.
