  One way to speed up the algorithm configuration task is to use short runs
instead of long runs as much as possible, but without discarding the
configurations that eventually do well on the long runs. We consider the
problem of selecting the top performing configurations of the Conditional
Markov Chain Search (CMCS), a general algorithm schema that includes, for
examples, VNS. We investigate how the structure of performance on short tests
links with those on long tests, showing that significant differences arise
between test domains. We propose a "performance envelope" method to exploit the
links; that learns when runs should be terminated, but that automatically
adapts to the domain.
