  Randomized experimentation (also known as A/B testing or bucket testing) is
widely used in the internet industry to measure the metric impact obtained by
different treatment variants. A/B tests identify the treatment variant showing
the best performance, which then becomes the chosen or selected treatment for
the entire population. However, the effect of a given treatment can differ
across experimental units and a personalized approach for treatment selection
can greatly improve upon the usual global selection strategy. In this work, we
develop a framework for personalization through (i) estimation of heterogeneous
treatment effect at either a cohort or member-level, followed by (ii) selection
of optimal treatment variants for cohorts (or members) obtained through
(deterministic or stochastic) constrained optimization.
  We perform a two-fold evaluation of our proposed methods. First, a simulation
analysis is conducted to study the effect of personalized treatment selection
under carefully controlled settings. This simulation illustrates the
differences between the proposed methods and the suitability of each with
increasing uncertainty. We also demonstrate the effectiveness of the method
through a real-life example related to serving notifications at Linkedin. The
solution significantly outperformed both heuristic solutions and the global
treatment selection baseline leading to a sizable win on top-line metrics like
member visits.
