  Observational studies are often used to understand relationships between
exposures and outcomes. They do not, however, allow conclusions about causal
relationships to be drawn unless statistical techniques are used to account for
the imbalance of confounders across exposure groups. Propensity score and
balance weighting (PSBW) are useful techniques that aim to reduce the
imbalances between exposure groups by weighting the groups to look alike on the
observed confounders. Despite the plethora of available methods to estimate
PSBW, there is little guidance on what one defines as adequate balance, and
unbiased and robust estimation of the causal treatment effect is not guaranteed
unless several conditions hold. Accurate inference requires that 1. the
treatment allocation mechanism is known, 2. the relationship between the
baseline covariates and the outcome is known, 3. adequate balance of baseline
covariates is achieved post-weighting, 4. a proper set of covariates to control
for confounding bias is known, and 5. a large enough sample size is available.
In this article, we use simulated data of various sizes to investigate the
influence of these five factors on statistical inference. Our findings provide
evidence that the maximum Kolmogorov- Smirnov statistic is the proper
statistical measure to assess balance on the baseline covariates, in contrast
to the mean standardised mean difference used in many applications, and 0.1 is
a suitable threshold to consider as acceptable balance. Finally, we recommend
that 60-80 observations, per confounder per treatment group, are required to
obtain a reliable and unbiased estimation of the causal treatment effect.
