  Discovering causal structure among a set of variables is a fundamental
problem in many domains. However, state-of-the-art methods seldom consider the
possibility that the observational data has missing values (incomplete data),
which is ubiquitous in many real-world situations. The missing value will
significantly impair the performance and even make the causal discovery
algorithms fail. In this paper, we propose an approach to discover causal
structures from incomplete data by using a novel encoder and reinforcement
learning (RL). The encoder is designed for missing data imputation as well as
feature extraction. In particular, it learns to encode the currently available
information (with missing values) into a robust feature representation which is
then used to determine where to search the best graph. The encoder is
integrated into a RL framework that can be optimized using the actor-critic
algorithm. Our method takes the incomplete observational data as input and
generates a causal structure graph. Experimental results on synthetic and real
data demonstrate that our method can robustly generate causal structures from
incomplete data. Compared with the direct combination of data imputation and
causal discovery methods, our method performs generally better and can even
obtain a performance gain as much as 43.2%.
