  With growing interest in Procedural Content Generation (PCG) it becomes
increasingly important to develop methods and tools for evaluating and
comparing alternative systems. There is a particular lack regarding the
evaluation of generative pipelines, where a set of generative systems work in
series to make iterative changes to an artifact. We introduce a novel method
called Generative Shift for evaluating the impact of individual stages in a PCG
pipeline by quantifying the impact that a generative process has when it is
applied to a pre-existing artifact. We explore this technique by applying it to
a very rich dataset of Minecraft game maps produced by a set of alternative
settlement generators developed as part of the Generative Design in Minecraft
Competition (GDMC), all of which are designed to produce appropriate
settlements for a pre-existing map. While this is an early exploration of this
technique we find it to be a promising lens to apply to PCG evaluation, and we
are optimistic about the potential of Generative Shift to be a domain-agnostic
method for evaluating generative pipelines.
