  The generative capabilities of deep learning neural networks (DNNs) have been
attracting increasing attention for both the remarkable artifacts they produce,
but also because of the vast conceptual difference between how they are
programmed and what they do. DNNs are 'black boxes' where high-level behavior
is not explicitly programmed, but emerges from the complex interactions of
thousands or millions of simple computational elements. Their behavior is often
described in anthropomorphic terms that can be misleading, seem magical, or
stoke fears of an imminent singularity in which machines become 'more' than
human. In this paper, we examine 5 distinct behavioral characteristics
associated with creativity, and provide an example of a mechanisms from
generative deep learning architectures that give rise to each these
characteristics. All 5 emerge from machinery built for purposes other than the
creative characteristics they exhibit, mostly classification. These mechanisms
of creative generative capabilities thus demonstrate a deep kinship to
computational perceptual processes. By understanding how these different
behaviors arise, we hope to on one hand take the magic out of anthropomorphic
descriptions, but on the other, to build a deeper appreciation of machinic
forms of creativity on their own terms that will allow us to nurture their
further development.
