  Real-world processes often contain intermediate state that can be modeled as
an extremely sparse tensor. We introduce Sparling, a technique that allows you
to learn models with intermediate layers that match this state from only
end-to-end labeled examples (i.e., no supervision on the intermediate state).
Sparling uses a new kind of informational bottleneck that enforces levels of
activation sparsity unachievable using other techniques. We find that extreme
sparsity is necessary to achieve good intermediate state modeling. On our
synthetic DigitCircle domain as well as the LaTeX-OCR and Audio-MNIST-Sequence
domains, we are able to precisely localize the intermediate states up to
feature permutation with > 90% accuracy, even though we only train end-to-end.
