  Multiple types of inference are available for probabilistic graphical models,
e.g., marginal, maximum-a-posteriori, and even marginal maximum-a-posteriori.
Which one do researchers mean when they talk about "planning as inference"?
There is no consistency in the literature, different types are used, and their
ability to do planning is further entangled with specific approximations or
additional constraints. In this work we use the variational framework to show
that all commonly used types of inference correspond to different weightings of
the entropy terms in the variational problem, and that planning corresponds
_exactly_ to a _different_ set of weights. This means that all the tricks of
variational inference are readily applicable to planning. We develop an
analogue of loopy belief propagation that allows us to perform approximate
planning in factored state Markov decisions processes without incurring
intractability due to the exponentially large state space. The variational
perspective shows that the previous types of inference for planning are only
adequate in environments with low stochasticity, and allows us to characterize
each type by its own merits, disentangling the type of inference from the
additional approximations that its practical use requires. We validate these
results empirically on synthetic MDPs and tasks posed in the International
Planning Competition.
