  Randomised control trials are currently the definitive gold standard approach
for formal educational experiments. Although conclusions from these experiments
are highly credible, their relatively slow experimentation rate, high expense
and rigid framework can be seen to limit scope on: 1. $\textit{metrics}$:
automation of the consistent rigorous computation of hundreds of metrics for
every experiment; 2. $\textit{concurrency}$: fast automated releases of
hundreds of concurrent experiments daily; and 3. $\textit{safeguards}$: safety
net tests and ramping up/rolling back treatments quickly to minimise negative
impact. This paper defines a framework for categorising different experimental
processes, and places a particular emphasis on technology readiness.
  On the basis of our analysis, our thesis is that the next generation of
education technology successes will be heralded by recognising the context of
experiments and collectively embracing the full set of processes that are at
hand: from rapid ideation and prototyping produced in small scale experiments
on the one hand, to influencing recommendations of best teaching practices with
large-scale and technology-enabled online A/B testing on the other. A key
benefit of the latter is that the running costs tend towards zero (leading to
`free experimentation'). This offers low-risk opportunities to explore and
drive value though well-planned lasting campaigns that iterate quickly at a
large scale. Importantly, because these experimental platforms are so
adaptable, the cumulative effect of the experimental campaign delivers
compounding value exponentially over time even if each individual experiment
delivers a small effect.
