  In this work we propose a model that can manipulate individual visual
attributes of objects in a real scene using examples of how respective
attribute manipulations affect the output of a simulation. As an example, we
train our model to manipulate the expression of a human face using
nonphotorealistic 3D renders of a face with varied expression. Our model
manages to preserve all other visual attributes of a real face, such as head
orientation, even though this and other attributes are not labeled in either
real or synthetic domain. Since our model learns to manipulate a specific
property in isolation using only "synthetic demonstrations" of such
manipulations without explicitly provided labels, it can be applied to shape,
texture, lighting, and other properties that are difficult to measure or
represent as real-valued vectors. We measure the degree to which our model
preserves other attributes of a real image when a single specific attribute is
manipulated. We use digit datasets to analyze how discrepancy in attribute
distributions affects the performance of our model, and demonstrate results in
a far more difficult setting: learning to manipulate real human faces using
nonphotorealistic 3D renders.
