  Learning under one-sided feedback (i.e., where we only observe the labels for
examples we predicted positively on) is a fundamental problem in machine
learning -- applications include lending and recommendation systems. Despite
this, there has been surprisingly little progress made in ways to mitigate the
effects of the sampling bias that arises. We focus on generalized linear models
and show that without adjusting for this sampling bias, the model may converge
suboptimally or even fail to converge to the optimal solution. We propose an
adaptive approach that comes with theoretical guarantees and show that it
outperforms several existing methods empirically. Our method leverages variance
estimation techniques to efficiently learn under uncertainty, offering a more
principled alternative compared to existing approaches.
