  Dependency parsing research, which has made significant gains in recent
years, typically focuses on improving the accuracy of single-tree predictions.
However, ambiguity is inherent to natural language syntax, and communicating
such ambiguity is important for error analysis and better-informed downstream
applications. In this work, we propose a transition sampling algorithm to
sample from the full joint distribution of parse trees defined by a
transition-based parsing model, and demonstrate the use of the samples in
probabilistic dependency analysis. First, we define the new task of dependency
path prediction, inferring syntactic substructures over part of a sentence, and
provide the first analysis of performance on this task. Second, we demonstrate
the usefulness of our Monte Carlo syntax marginal method for parser error
analysis and calibration. Finally, we use this method to propagate parse
uncertainty to two downstream information extraction applications: identifying
persons killed by police and semantic role assignment.
