  In the causal learning setting, we wish to learn cause-and-effect
relationships between variables such that we can correctly infer the effect of
an intervention. While the difference between a cyclic structure and an acyclic
structure may be just a single edge, cyclic causal structures have
qualitatively different behavior under intervention: cycles cause feedback
loops when the downstream effect of an intervention propagates back to the
source variable. We present three theoretical observations about probability
distributions with self-referential factorizations, i.e. distributions that
could be graphically represented with a cycle. First, we prove that
self-referential distributions in two variables are, in fact, independent.
Second, we prove that self-referential distributions in N variables have zero
mutual information. Lastly, we prove that self-referential distributions that
factorize in a cycle, also factorize as though the cycle were reversed. These
results suggest that cyclic causal dependence may exist even where
observational data suggest independence among variables. Methods based on
estimating mutual information, or heuristics based on independent causal
mechanisms, are likely to fail to learn cyclic casual structures. We encourage
future work in causal learning that carefully considers cycles.
