  We address a fundamental problem in chemistry known as chemical reaction
product prediction. Our main insight is that the input reactant and reagent
molecules can be jointly represented as a graph, and the process of generating
product molecules from reactant molecules can be formulated as a sequence of
graph transformations. To this end, we propose Graph Transformation Policy
Network (GTPN) -- a novel generic method that combines the strengths of graph
neural networks and reinforcement learning to learn the reactions directly from
data with minimal chemical knowledge. Compared to previous methods, GTPN has
some appealing properties such as: end-to-end learning, and making no
assumption about the length or the order of graph transformations. In order to
guide model search through the complex discrete space of sets of bond changes
effectively, we extend the standard policy gradient loss by adding useful
constraints. Evaluation results show that GTPN improves the top-1 accuracy over
the current state-of-the-art method by about 3% on the large USPTO dataset. Our
model's performances and prediction errors are also analyzed carefully in the
paper.
