  We developed Distilled Graph Attention Policy Network (DGAPN), a
reinforcement learning model to generate novel graph-structured chemical
representations that optimize user-defined objectives by efficiently navigating
a physically constrained domain. The framework is examined on the task of
generating molecules that are designed to bind, noncovalently, to functional
sites of SARS-CoV-2 proteins. We present a spatial Graph Attention (sGAT)
mechanism that leverages self-attention over both node and edge attributes as
well as encoding the spatial structure -- this capability is of considerable
interest in synthetic biology and drug discovery. An attentional policy network
is introduced to learn the decision rules for a dynamic, fragment-based
chemical environment, and state-of-the-art policy gradient techniques are
employed to train the network with stability. Exploration is driven by the
stochasticity of the action space design and the innovation reward bonuses
learned and proposed by random network distillation. In experiments, our
framework achieved outstanding results compared to state-of-the-art algorithms,
while reducing the complexity of paths to chemical synthesis.
