  When designing experimental studies with human participants, experimenters
must decide how many trials each participant will complete, as well as how many
participants to test. Most discussion of statistical power (the ability of a
study design to detect an effect) has focussed on sample size, and assumed
sufficient trials. Here we explore the influence of both factors on statistical
power, represented as a two-dimensional plot on which iso-power contours can be
visualised. We demonstrate the conditions under which the number of trials is
particularly important, i.e. when the within-participant variance is large
relative to the between-participants variance. We then derive power contour
plots using existing data sets for eight experimental paradigms and
methodologies (including reaction times, sensory thresholds, fMRI, MEG, and
EEG), and provide example code to calculate estimates of the within- and
between-participant variance for each method. In all cases, the
within-participant variance was larger than the between-participants variance,
meaning that the number of trials has a meaningful influence on statistical
power in commonly used paradigms. An online tool is provided
(https://shiny.york.ac.uk/powercontours/) for generating power contours, from
which the optimal combination of trials and participants can be calculated when
designing future studies.
