  Prior work has shown that Visual Recognition datasets frequently
underrepresent bias groups $B$ (\eg Female) within class labels $Y$ (\eg
Programmers). This dataset bias can lead to models that learn spurious
correlations between class labels and bias groups such as age, gender, or race.
Most recent methods that address this problem require significant architectural
changes or additional loss functions requiring more hyper-parameter tuning.
Alternatively, data sampling baselines from the class imbalance literature (\eg
Undersampling, Upweighting), which can often be implemented in a single line of
code and often have no hyperparameters, offer a cheaper and more efficient
solution. However, these methods suffer from significant shortcomings. For
example, Undersampling drops a significant part of the input distribution per
epoch while Oversampling repeats samples, causing overfitting. To address these
shortcomings, we introduce a new class-conditioned sampling method: Bias
Mimicking. The method is based on the observation that if a class $c$ bias
distribution, \ie $P_D(B|Y=c)$ is mimicked across every $c^{\prime}\neq c$,
then $Y$ and $B$ are statistically independent. Using this notion, BM, through
a novel training procedure, ensures that the model is exposed to the entire
distribution per epoch without repeating samples. Consequently, Bias Mimicking
improves underrepresented groups' accuracy of sampling methods by 3\% over four
benchmarks while maintaining and sometimes improving performance over
nonsampling methods. Code: \url{https://github.com/mqraitem/Bias-Mimicking}
