  Robust estimation under Huber's $\epsilon$-contamination model has become an
important topic in statistics and theoretical computer science. Statistically
optimal procedures such as Tukey's median and other estimators based on depth
functions are impractical because of their computational intractability. In
this paper, we establish an intriguing connection between $f$-GANs and various
depth functions through the lens of $f$-Learning. Similar to the derivation of
$f$-GANs, we show that these depth functions that lead to statistically optimal
robust estimators can all be viewed as variational lower bounds of the total
variation distance in the framework of $f$-Learning. This connection opens the
door of computing robust estimators using tools developed for training GANs. In
particular, we show in both theory and experiments that some appropriate
structures of discriminator networks with hidden layers in GANs lead to
statistically optimal robust location estimators for both Gaussian distribution
and general elliptical distributions where first moment may not exist.
