  Randomized field experiments are the gold standard for evaluating the impact
of software changes on customers. In the online domain, randomization has been
the main tool to ensure exchangeability. However, due to the different
deployment conditions and the high dependence on the surrounding environment,
designing experiments for automotive software needs to consider a higher number
of restricted variables to ensure conditional exchangeability. In this paper,
we show how at Volvo Cars we utilize causal graphical models to design
experiments and explicitly communicate the assumptions of experiments. These
graphical models are used to further assess the experiment validity, compute
direct and indirect causal effects, and reason on the transportability of the
causal conclusions.
