  Real-time bidding (RTB) has become a major paradigm of display advertising.
Each ad impression generated from a user visit is auctioned in real time, where
demand-side platform (DSP) automatically provides bid price usually relying on
the ad impression value estimation and the optimal bid price determination.
However, the current bid strategy overlooks large randomness of the user
behaviors (e.g., click) and the cost uncertainty caused by the auction
competition. In this work, we explicitly factor in the uncertainty of estimated
ad impression values and model the risk preference of a DSP under a specific
state and market environment via a sequential decision process. Specifically,
we propose a novel adaptive risk-aware bidding algorithm with budget constraint
via reinforcement learning, which is the first to simultaneously consider
estimation uncertainty and the dynamic risk tendency of a DSP. We theoretically
unveil the intrinsic relation between the uncertainty and the risk tendency
based on value at risk (VaR). Consequently, we propose two instantiations to
model risk tendency, including an expert knowledge-based formulation embracing
three essential properties and an adaptive learning method based on
self-supervised reinforcement learning. We conduct extensive experiments on
public datasets and show that the proposed framework outperforms
state-of-the-art methods in practical settings.
