  Bidding strategies that help advertisers determine bidding prices are
receiving increasing attention as more and more ad impressions are sold through
real-time bidding systems. This paper first describes the problem and
challenges of optimizing bidding strategies for individual advertisers in
real-time bidding display advertising. Then, several representative bidding
strategies are introduced, especially the research advances and challenges of
reinforcement learning-based bidding strategies. Further, we quantitatively
evaluate the performance of several representative bidding strategies on the
iPinYou dataset. Specifically, we examine the effects of state, action, and
reward function on the performance of reinforcement learning-based bidding
strategies. Finally, we summarize the general steps for optimizing bidding
strategies using reinforcement learning algorithms and present our suggestions.
