  Commonly in reinforcement learning (RL), rewards are discounted over time
using an exponential function to model time preference, thereby bounding the
expected long-term reward. In contrast, in economics and psychology, it has
been shown that humans often adopt a hyperbolic discounting scheme, which is
optimal when a specific task termination time distribution is assumed. In this
work, we propose a theory for continuous-time model-based reinforcement
learning generalized to arbitrary discount functions. This formulation covers
the case in which there is a non-exponential random termination time. We derive
a Hamilton-Jacobi-Bellman (HJB) equation characterizing the optimal policy and
describe how it can be solved using a collocation method, which uses deep
learning for function approximation. Further, we show how the inverse RL
problem can be approached, in which one tries to recover properties of the
discount function given decision data. We validate the applicability of our
proposed approach on two simulated problems. Our approach opens the way for the
analysis of human discounting in sequential decision-making tasks.
