  Meta-training agents with memory has been shown to culminate in Bayes-optimal
agents, which casts Bayes-optimality as the implicit solution to a numerical
optimization problem rather than an explicit modeling assumption. Bayes-optimal
agents are risk-neutral, since they solely attune to the expected return, and
ambiguity-neutral, since they act in new situations as if the uncertainty were
known. This is in contrast to risk-sensitive agents, which additionally exploit
the higher-order moments of the return, and ambiguity-sensitive agents, which
act differently when recognizing situations in which they lack knowledge.
Humans are also known to be averse to ambiguity and sensitive to risk in ways
that aren't Bayes-optimal, indicating that such sensitivity can confer
advantages, especially in safety-critical situations. How can we extend the
meta-learning protocol to generate risk- and ambiguity-sensitive agents? The
goal of this work is to fill this gap in the literature by showing that risk-
and ambiguity-sensitivity also emerge as the result of an optimization problem
using modified meta-training algorithms, which manipulate the
experience-generation process of the learner. We empirically test our proposed
meta-training algorithms on agents exposed to foundational classes of
decision-making experiments and demonstrate that they become sensitive to risk
and ambiguity.
