  We seek to automate the design of molecules based on specific chemical
properties. Our primary contributions are a simpler method for generating
SMILES strings guaranteed to be chemically valid, using a combination of a new
context-free grammar for SMILES and additional masking logic; and casting the
molecular property optimization as a reinforcement learning problem,
specifically best-of-batch policy gradient applied to a Transformer model
architecture. This approach uses substantially fewer model steps per atom than
earlier approaches, thus enabling generation of larger molecules, and beats
previous state-of-the art baselines by a significant margin. Applying
reinforcement learning to a combination of a custom context-free grammar with
additional masking to enforce non-local constraints is applicable to any
optimization of a graph structure under a mixture of local and nonlocal
constraints.
