  When estimating a regression model, we might have data where some labels are
missing, or our data might be biased by a selection mechanism. When the
response or selection mechanism is ignorable (i.e., independent of the response
variable given the features) one can use off-the-shelf regression methods; in
the nonignorable case one typically has to adjust for bias. We observe that
privileged information (i.e. information that is only available during
training) might render a nonignorable selection mechanism ignorable, and we
refer to this scenario as Privilegedly Missing at Random (PMAR). We propose a
novel imputation-based regression method, named repeated regression, that is
suitable for PMAR. We also consider an importance weighted regression method,
and a doubly robust combination of the two. The proposed methods are easy to
implement with most popular out-of-the-box regression algorithms. We
empirically assess the performance of the proposed methods with extensive
simulated experiments and on a synthetically augmented real-world dataset. We
conclude that repeated regression can appropriately correct for bias, and can
have considerable advantage over weighted regression, especially when
extrapolating to regions of the feature space where response is never observed.
