  The Probability Ranking Principle (PRP) has been considered as the
foundational standard in the design of information retrieval (IR) systems. The
principle requires an IR module's returned list of results to be ranked with
respect to the underlying user interests, so as to maximize the results'
utility.
  Nevertheless, we point out that it is inappropriate to indiscriminately apply
PRP through every stage of a contemporary IR system. Such systems contain
multiple stages (e.g., retrieval, pre-ranking, ranking, and re-ranking stages,
as examined in this paper). The \emph{selection bias} inherent in the model of
each stage significantly influences the results that are ultimately presented
to users.
  To address this issue, we propose an improved ranking principle for
multi-stage systems, namely the Generalized Probability Ranking Principle
(GPRP), to emphasize both the selection bias in each stage of the system
pipeline as well as the underlying interest of users.
  We realize GPRP via a unified algorithmic framework named Full Stage Learning
to Rank. Our core idea is to first estimate the selection bias in the
subsequent stages and then learn a ranking model that best complies with the
downstream modules' selection bias so as to deliver its top ranked results to
the final ranked list in the system's output.
  We performed extensive experiment evaluations of our developed Full Stage
Learning to Rank solution, using both simulations and online A/B tests in one
of the leading short-video recommendation platforms. The algorithm is proved to
be effective in both retrieval and ranking stages. Since deployed, the
algorithm has brought consistent and significant performance gain to the
platform.
