  Offline reinforcement-learning (RL) algorithms learn to make decisions using
a given, fixed training dataset without online data collection. This problem
setting is captivating because it holds the promise of utilizing previously
collected datasets without any costly or risky interaction with the
environment. However, this promise also bears the drawback of this setting as
the restricted dataset induces uncertainty because the agent can encounter
unfamiliar sequences of states and actions that the training data did not
cover. To mitigate the destructive uncertainty effects, we need to balance the
aspiration to take reward-maximizing actions with the incurred risk due to
incorrect ones. In financial economics, modern portfolio theory (MPT) is a
method that risk-averse investors can use to construct diversified portfolios
that maximize their returns without unacceptable levels of risk. We propose
integrating MPT into the agent's decision-making process, presenting a new
simple-yet-highly-effective risk-aware planning algorithm for offline RL. Our
algorithm allows us to systematically account for the \emph{estimated quality}
of specific actions and their \emph{estimated risk} due to the uncertainty. We
show that our approach can be coupled with the Transformer architecture to
yield a state-of-the-art planner, which maximizes the return for offline RL
tasks. Moreover, our algorithm reduces the variance of the results
significantly compared to conventional Transformer decoding, which results in a
much more stable algorithm -- a property that is essential for the offline RL
setting, where real-world exploration and failures can be costly or dangerous.
