  These notes aim to shed light on the recently proposed structured projected
intermediate gradient optimization technique (SPIGOT, Peng et al., 2018).
SPIGOT is a variant of the straight-through estimator (Bengio et al., 2013)
which bypasses gradients of the argmax function by back-propagating a surrogate
"gradient." We provide a new interpretation to the proposed gradient and put
this technique into perspective, linking it to other methods for training
neural networks with discrete latent variables. As a by-product, we suggest
alternate variants of SPIGOT which will be further explored in future work.
