  In structured prediction problems where we have indirect supervision of the
output, maximum marginal likelihood faces two computational obstacles:
non-convexity of the objective and intractability of even a single gradient
computation. In this paper, we bypass both obstacles for a class of what we
call linear indirectly-supervised problems. Our approach is simple: we solve a
linear system to estimate sufficient statistics of the model, which we then use
to estimate parameters via convex optimization. We analyze the statistical
properties of our approach and show empirically that it is effective in two
settings: learning with local privacy constraints and learning from low-cost
count-based annotations.
