  Answering counterfactual queries has important applications such as
explainability, robustness, and fairness but is challenging when the causal
variables are unobserved and the observations are non-linear mixtures of these
latent variables, such as pixels in images. One approach is to recover the
latent Structural Causal Model (SCM), which may be infeasible in practice due
to requiring strong assumptions, e.g., linearity of the causal mechanisms or
perfect atomic interventions. Meanwhile, more practical ML-based approaches
using naive domain translation models to generate counterfactual samples lack
theoretical grounding and may construct invalid counterfactuals. In this work,
we strive to strike a balance between practicality and theoretical guarantees
by analyzing a specific type of causal query called domain counterfactuals,
which hypothesizes what a sample would have looked like if it had been
generated in a different domain (or environment). We show that recovering the
latent SCM is unnecessary for estimating domain counterfactuals, thereby
sidestepping some of the theoretic challenges. By assuming invertibility and
sparsity of intervention, we prove domain counterfactual estimation error can
be bounded by a data fit term and intervention sparsity term. Building upon our
theoretical results, we develop a theoretically grounded practical algorithm
that simplifies the modeling process to generative model estimation under
autoregressive and shared parameter constraints that enforce intervention
sparsity. Finally, we show an improvement in counterfactual estimation over
baseline methods through extensive simulated and image-based experiments.
