  Representing a space of handwriting stroke styles includes the challenge of
representing both the style of each character and the overall style of the
human writer. Existing VRNN approaches to representing handwriting often do not
distinguish between these different style components, which can reduce model
capability. Instead, we introduce the Decoupled Style Descriptor (DSD) model
for handwriting, which factors both character- and writer-level styles and
allows our model to represent an overall greater space of styles. This approach
also increases flexibility: given a few examples, we can generate handwriting
in new writer styles, and also now generate handwriting of new characters
across writer styles. In experiments, our generated results were preferred over
a state of the art baseline method 88% of the time, and in a writer
identification task on 20 held-out writers, our DSDs achieved 89.38% accuracy
from a single sample word. Overall, DSDs allows us to improve both the quality
and flexibility over existing handwriting stroke generation approaches.
