  Measuring treatment effects in observational studies is challenging because
of confounding bias. Confounding occurs when a variable affects both the
treatment and the outcome. Traditional methods such as propensity score
matching estimate treatment effects by conditioning on the confounders. Recent
literature has presented new methods that use machine learning to predict the
counterfactuals in observational studies which then allow for estimating
treatment effects. These studies however, have been applied to real world data
where the true treatment effects have not been known. This study aimed to study
the effectiveness of this counterfactual prediction method by simulating two
main scenarios: with and without confounding. Each type also included linear
and non-linear relationships between input and output data. The key item in the
simulations was that we generated known true causal effects. Linear regression,
lasso regression and random forest models were used to predict the
counterfactuals and treatment effects. These were compared these with the true
treatment effect as well as a naive treatment effect. The results show that the
most important factor in whether this machine learning method performs well, is
the degree of non-linearity in the data. Surprisingly, for both non-confounding
\textit{and} confounding, the machine learning models all performed well on the
linear dataset. However, when non-linearity was introduced, the models
performed very poorly. Therefore under the conditions of this simulation study,
the machine learning method performs well under conditions of linearity, even
if confounding is present, but at this stage should not be trusted when
non-linearity is introduced.
