  Disentangling the encodings of neural models is a fundamental aspect for
improving interpretability, semantic control and downstream task performance in
Natural Language Processing. Currently, most disentanglement methods are
unsupervised or rely on synthetic datasets with known generative factors. We
argue that recurrent syntactic and semantic regularities in textual data can be
used to provide the models with both structural biases and generative factors.
We leverage the semantic structures present in a representative and
semantically dense category of sentence types, definitional sentences, for
training a Variational Autoencoder to learn disentangled representations. Our
experimental results show that the proposed model outperforms unsupervised
baselines on several qualitative and quantitative benchmarks for
disentanglement, and it also improves the results in the downstream task of
definition modeling.
