  We consider the task of learning causal structures from data stored on
multiple machines, and propose a novel structure learning method called
distributed annealing on regularized likelihood score (DARLS) to solve this
problem. We model causal structures by a directed acyclic graph that is
parameterized with generalized linear models, so that our method is applicable
to various types of data. To obtain a high-scoring causal graph, DARLS
simulates an annealing process to search over the space of topological sorts,
where the optimal graphical structure compatible with a sort is found by a
distributed optimization method. This distributed optimization relies on
multiple rounds of communication between local and central machines to estimate
the optimal structure. We establish its convergence to a global optimizer of
the overall score that is computed on all data across local machines. To the
best of our knowledge, DARLS is the first distributed method for learning
causal graphs with such theoretical guarantees. Through extensive simulation
studies, DARLS has shown competing performance against existing methods on
distributed data, and achieved comparable structure learning accuracy and
test-data likelihood with competing methods applied to pooled data across all
local machines. In a real-world application for modeling protein-DNA binding
networks with distributed ChIP-Sequencing data, DARLS also exhibits higher
predictive power than other methods, demonstrating a great advantage in
estimating causal networks from distributed data.
