  Ranking is a crucial module using in the recommender system. In particular,
the ranking module using in our YoungTao recommendation scenario is to provide
an ordered list of items to users, to maximize the click number throughout the
recommendation session for each user. However, we found that the traditional
ranking method for optimizing Click-Through rate(CTR) cannot address our
ranking scenario well, since it completely ignores user leaving, and CTR is the
optimization goal for the one-step recommendation. To effectively undertake the
purpose of our ranking module, we propose a long-term optimization goal, named
as CTE (Click-Through quantity expectation), for explicitly taking the behavior
of user leaving into account. Based on CTE, we propose an effective model
trained by reinforcement learning. Moreover, we build a simulation environment
from offline log data for estimating PBR and CTR. We conduct extensive
experiments on offline datasets and an online e-commerce scenario in TaoBao.
Experimental results show that our method can boost performance effectively
