  A/B testing is a widely-used paradigm within marketing optimization because
it promises identification of causal effects and because it is implemented out
of the box in most messaging delivery software platforms. Modern businesses,
however, often run many A/B/n tests at the same time and in parallel, and
package many content variations into the same messages, not all of which are
part of an explicit test. Whether as the result of many teams testing at the
same time, or as part of a more sophisticated reinforcement learning (RL)
approach that continuously adapts tests and test condition assignment based on
previous results, dynamic parallel testing cannot be evaluated the same way
traditional A/B tests are evaluated. This paper presents a method for
disentangling the causal effects of the various tests under conditions of
continuous test adaptation, using a matched-synthetic control group that adapts
alongside the tests.
