  Datasets with significant proportions of bias present threats for training a
trustworthy model on NLU tasks. Despite yielding great progress, current
debiasing methods impose excessive reliance on the knowledge of bias
attributes. Definition of the attributes, however, is elusive and varies across
different datasets. Furthermore, leveraging these attributes at input level to
bias mitigation may leave a gap between intrinsic properties and the underlying
decision rule. To narrow down this gap and liberate the supervision on bias, we
suggest extending bias mitigation into feature space. Therefore, a novel model,
Recovering Intended-Feature Subspace with Knowledge-Free (RISK) is developed.
Assuming that shortcut features caused by various biases are unintended for
prediction, RISK views them as redundant features. When delving into a lower
manifold to remove redundancies, RISK reveals that an extremely low-dimensional
subspace with intended features can robustly represent the highly biased
dataset. Empirical results demonstrate our model can consistently improve model
generalization to out-of-distribution set, and achieves a new state-of-the-art
performance.
