  Conversion rate optimization means designing web interfaces such that more
visitors perform a desired action (such as register or purchase) on the site.
One promising approach, implemented in Sentient Ascend, is to optimize the
design using evolutionary algorithms, evaluating each candidate design online
with actual visitors. Because such evaluations are costly and noisy, several
challenges emerge: How can available visitor traffic be used most efficiently?
How can good solutions be identified most reliably? How can a high conversion
rate be maintained during optimization? This paper proposes a new technique to
address these issues. Traffic is allocated to candidate solutions using a
multi-armed bandit algorithm, using more traffic on those evaluations that are
most useful. In a best-arm identification mode, the best candidate can be
identified reliably at the end of evolution, and in a campaign mode, the
overall conversion rate can be optimized throughout the entire evolution
process. Multi-armed bandit algorithms thus improve performance and reliability
of machine discovery in noisy real-world environments.
