  Counterfactual analysis is intuitively performed by humans on a daily basis
eg. "What should I have done differently to get the loan approved?". Such
counterfactual questions also steer the formulation of scientific hypotheses.
More formally it provides insights about potential improvements of a system by
inferring the effects of hypothetical interventions into a past observation of
the system's behaviour which plays a prominent role in a variety of industrial
applications. Due to the hypothetical nature of such analysis, counterfactual
distributions are inherently ambiguous. This ambiguity is particularly
challenging in continuous settings in which a continuum of explanations exist
for the same observation. In this paper, we address this problem by following a
hierarchical Bayesian approach which explicitly models such uncertainty. In
particular, we derive counterfactual distributions for a Bayesian Warped
Gaussian Process thereby allowing for non-Gaussian distributions and
non-additive noise. We illustrate the properties our approach on a synthetic
and on a semi-synthetic example and show its performance when used within an
algorithmic recourse downstream task.
