  The intricate hierarchical structure of syntax is fundamental to the
intricate and systematic nature of human language. This study investigates the
premise that language models, specifically their attention distributions, can
encapsulate syntactic dependencies. We introduce Dynamic Syntax Mapping (DSM),
an innovative approach for the agnostic induction of these structures. Our
method diverges from traditional syntax models which rely on predefined
annotation schemata. Instead, we focus on a core characteristic inherent in
dependency relations: syntactic substitutability. This concept refers to the
interchangeability of words within the same syntactic category at either end of
a dependency. By leveraging this property, we generate a collection of
syntactically invariant sentences, which serve as the foundation for our
parsing framework. Our findings reveal that the use of an increasing array of
substitutions notably enhances parsing precision on natural language data.
Specifically, in the context of long-distance subject-verb agreement, DSM
exhibits a remarkable advancement over prior methodologies. Furthermore, DSM's
adaptability is demonstrated through its successful application in varied
parsing scenarios, underscoring its broad applicability.
