  We consider the problem of bid prediction in repeated auctions and evaluate
the performance of econometric methods for learning agents using a dataset from
a mainstream sponsored search auction marketplace. Sponsored search auctions is
a billion dollar industry and the main source of revenue of several tech
giants. A critical problem in optimizing such marketplaces is understanding how
bidders will react to changes in the auction design. We propose the use of
no-regret based econometrics for bid prediction, modeling players as no-regret
learners with respect to a utility function, unknown to the analyst. We propose
new econometric approaches to simultaneously learn the parameters of a player's
utility and her learning rule, and apply these methods in a real-world dataset
from the BingAds sponsored search auction marketplace. We show that the
no-regret econometric methods perform comparable to state-of-the-art
time-series machine learning methods when there is no co-variate shift, but
significantly outperform machine learning methods when there is a co-variate
shift between the training and test periods. This portrays the importance of
using structural econometric approaches in predicting how players will respond
to changes in the market. Moreover, we show that among structural econometric
methods, approaches based on no-regret learning outperform more traditional,
equilibrium-based, econometric methods that assume that players continuously
best-respond to competition. Finally, we demonstrate how the prediction
performance of the no-regret learning algorithms can be further improved by
considering bidders who optimize a utility function with a visibility bias
component.
