  Optimal stopping is the problem of deciding the right time at which to take a
particular action in a stochastic system, in order to maximize an expected
reward. It has many applications in areas such as finance, healthcare, and
statistics. In this paper, we employ deep Reinforcement Learning (RL) to learn
optimal stopping policies in two financial engineering applications: namely
option pricing, and optimal option exercise. We present for the first time a
comprehensive empirical evaluation of the quality of optimal stopping policies
identified by three state of the art deep RL algorithms: double deep Q-learning
(DDQN), categorical distributional RL (C51), and Implicit Quantile Networks
(IQN). In the case of option pricing, our findings indicate that in a
theoretical Black-Schole environment, IQN successfully identifies nearly
optimal prices. On the other hand, it is slightly outperformed by C51 when
confronted to real stock data movements in a put option exercise problem that
involves assets from the S&P500 index. More importantly, the C51 algorithm is
able to identify an optimal stopping policy that achieves 8% more out-of-sample
returns than the best of four natural benchmark policies. We conclude with a
discussion of our findings which should pave the way for relevant future
research.
