  Retrosynthetic planning is a fundamental problem in chemistry for finding a
pathway of reactions to synthesize a target molecule. Recently, search
algorithms have shown promising results for solving this problem by using deep
neural networks (DNNs) to expand their candidate solutions, i.e., adding new
reactions to reaction pathways. However, the existing works on this line are
suboptimal; the retrosynthetic planning problem requires the reaction pathways
to be (a) represented by real-world reactions and (b) executable using
"building block" molecules, yet the DNNs expand reaction pathways without fully
incorporating such requirements. Motivated by this, we propose an end-to-end
framework for directly training the DNNs towards generating reaction pathways
with the desirable properties. Our main idea is based on a self-improving
procedure that trains the model to imitate successful trajectories found by
itself. We also propose a novel reaction augmentation scheme based on a forward
reaction model. Our experiments demonstrate that our scheme significantly
improves the success rate of solving the retrosynthetic problem from 86.84% to
96.32% while maintaining the performance of DNN for predicting valid reactions.
