  Under stringent model type and variable distribution assumptions,
differentiable score-based causal discovery methods learn a directed acyclic
graph (DAG) from observational data by evaluating candidate graphs over an
average score function. Despite great success in low-dimensional linear
systems, it has been observed that these approaches overly exploit
easier-to-fit samples, thus inevitably learning spurious edges. Worse still,
inherent mostly in these methods the common homogeneity assumption can be
easily violated, due to the widespread existence of heterogeneous data in the
real world, resulting in performance vulnerability when noise distributions
vary. We propose a simple yet effective model-agnostic framework to boost
causal discovery performance by dynamically learning the adaptive weights for
the Reweighted Score function, ReScore for short, where the weights tailor
quantitatively to the importance degree of each sample. Intuitively, we
leverage the bilevel optimization scheme to \wx{alternately train a standard
DAG learner and reweight samples -- that is, upweight the samples the learner
fails to fit and downweight the samples that the learner easily extracts the
spurious information from. Extensive experiments on both synthetic and
real-world datasets are carried out to validate the effectiveness of ReScore.
We observe consistent and significant boosts in structure learning performance.
Furthermore, we visualize that ReScore concurrently mitigates the influence of
spurious edges and generalizes to heterogeneous data. Finally, we perform the
theoretical analysis to guarantee the structure identifiability and the weight
adaptive properties of ReScore in linear systems. Our codes are available at
https://github.com/anzhang314/ReScore.
