  Backtracking search algorithms are often used to solve the Constraint
Satisfaction Problem (CSP). The efficiency of backtracking search depends
greatly on the variable ordering heuristics. Currently, the most commonly used
heuristics are hand-crafted based on expert knowledge. In this paper, we
propose a deep reinforcement learning based approach to automatically discover
new variable ordering heuristics that are better adapted for a given class of
CSP instances. We show that directly optimizing the search cost is hard for
bootstrapping, and propose to optimize the expected cost of reaching a leaf
node in the search tree. To capture the complex relations among the variables
and constraints, we design a representation scheme based on Graph Neural
Network that can process CSP instances with different sizes and constraint
arities. Experimental results on random CSP instances show that the learned
policies outperform classical hand-crafted heuristics in terms of minimizing
the search tree size, and can effectively generalize to instances that are
larger than those used in training.
