  Variety of machine learning problems can be formulated as an optimization
task for some (surrogate) loss function. Calculation of loss function can be
viewed in terms of stochastic computation graphs (SCG). We use this formalism
to analyze a problem of optimization of famous sequence-to-sequence model with
attention and propose reformulation of the task. Examples are given for machine
translation (MT). Our work provides a unified view on different optimization
approaches for sequence-to-sequence models and could help researchers in
developing new network architectures with embedded stochastic nodes.
