  We present a novel podcast recommender system deployed at industrial scale.
This system successfully optimizes personal listening journeys that unfold over
months for hundreds of millions of listeners. In deviating from the pervasive
industry practice of optimizing machine learning algorithms for short-term
proxy metrics, the system substantially improves long-term performance in A/B
tests. The paper offers insights into how our methods cope with attribution,
coordination, and measurement challenges that usually hinder such long-term
optimization. To contextualize these practical insights within a broader
academic framework, we turn to reinforcement learning (RL). Using the language
of RL, we formulate a comprehensive model of users' recurring relationships
with a recommender system. Then, within this model, we identify our approach as
a policy improvement update to a component of the existing recommender system,
enhanced by tailored modeling of value functions and user-state
representations. Illustrative offline experiments suggest this specialized
modeling reduces data requirements by as much as a factor of 120,000 compared
to black-box approaches.
