  Real-time bidding, as one of the most popular mechanisms for selling online
ad slots, facilitates advertisers to reach their potential customers. The goal
of bidding optimization is to maximize the advertisers' return on investment
(ROI) under a certain budget setting. A straightforward solution is to model
the bidding function in an explicit form. However, the static functional
solutions lack generality in practice and are insensitive to the stochastic
behaviour of other bidders in the environment. In this paper, we propose a
general multi-agent framework with actor-critic solutions facing against
playing imperfect information games. We firstly introduce a novel Deep
Attentive Survival Analysis (DASA) model to infer the censored data in the
second price auctions which outperforms start-of-the-art survival analysis.
Furthermore, our approach introduces the DASA model as the opponent model into
the policy learning process for each agent and develop a mean field equilibrium
analysis of the second price auctions. The experiments have shown that with the
inference of the market, the market converges to the equilibrium much faster
while playing against both fixed strategy agents and dynamic learning agents.
