  Down-sampling training data has long been shown to improve the generalization
performance of a wide range of machine learning systems. Recently,
down-sampling has proved effective in genetic programming (GP) runs that
utilize the lexicase parent selection technique. Although this down-sampling
procedure has been shown to significantly improve performance across a variety
of problems, it does not seem to do so due to encouraging adaptability through
environmental change. We hypothesize that the random sampling that is performed
every generation causes discontinuities that result in the population being
unable to adapt to the shifting environment. We investigate modifications to
down-sampled lexicase selection in hopes of promoting incremental environmental
change to scaffold evolution by reducing the amount of jarring discontinuities
between the environments of successive generations. In our empirical studies,
we find that forcing incremental environmental change is not significantly
better for evolving solutions to program synthesis problems than simple random
down-sampling. In response to this, we attempt to exacerbate the hypothesized
prevalence of discontinuities by using only disjoint down-samples to see if it
hinders performance. We find that this also does not significantly differ from
the performance of regular random down-sampling. These negative results raise
new questions about the ways in which the composition of sub-samples, which may
include synonymous cases, may be expected to influence the performance of
machine learning systems that use down-sampling.
