  Discrete structure rules for validating molecular structures are usually
limited to fulfillment of the octet rule or similar simple deterministic
heuristics. We propose a model, inspired by language modeling from natural
language processing, with the ability to learn from a collection of undirected
molecular graphs, enabling fitting of any underlying structure rule present in
the collection. We introduce an adaption to the popular Transformer model,
which can learn relationships between atoms and bonds. To our knowledge, the
Transformer adaption is the first model that is trained to solve the
unsupervised task of recovering partially observed molecules. In this work, we
assess how different degrees of information impact performance w.r.t. to
fitting the QM9 dataset, which conforms to the octet rule, and to fitting the
ZINC dataset, which contains hypervalent molecules and ions requiring the model
to learn a more complex structure rule. More specifically, we test a full
discrete graph with bond order information, a full discrete graph with only
connectivity, a bag-of-neighbors, a bag-of-atoms, and a count-based unigram
statistics. These results provide encouraging evidence that neural networks,
even when only connectivity is available, can learn arbitrary molecular
structure rules specific to a dataset, as the Transformer adaption surpasses a
strong octet rule baseline on the ZINC dataset.
