Bounded Rational Decision-Making in Changing Environments
  A perfectly rational decision-maker chooses the best action with the highest
utility gain from a set of possible actions. The optimality principles that
describe such decision processes do not take into account the computational
costs of finding the optimal action. Bounded rational decision-making addresses
this problem by specifically trading off information-processing costs and
expected utility. Interestingly, a similar trade-off between energy and entropy
arises when describing changes in thermodynamic systems. This similarity has
been recently used to describe bounded rational agents. Crucially, this
framework assumes that the environment does not change while the decision-maker
is computing the optimal policy. When this requirement is not fulfilled, the
decision-maker will suffer inefficiencies in utility, that arise because the
current policy is optimal for an environment in the past. Here we borrow
concepts from non-equilibrium thermodynamics to quantify these inefficiencies
and illustrate with simulations its relationship with computational resources.
