  We develop an approach for solving time-consistent risk-sensitive stochastic
optimization problems using model-free reinforcement learning (RL).
Specifically, we assume agents assess the risk of a sequence of random
variables using dynamic convex risk measures. We employ a time-consistent
dynamic programming principle to determine the value of a particular policy,
and develop policy gradient update rules that aid in obtaining optimal
policies. We further develop an actor-critic style algorithm using neural
networks to optimize over policies. Finally, we demonstrate the performance and
flexibility of our approach by applying it to three optimization problems:
statistical arbitrage trading strategies, financial hedging, and obstacle
avoidance robot control.
