  Machine learning has demonstrated remarkable prediction accuracy over i.i.d
data, but the accuracy often drops when tested with data from another
distribution. In this paper, we aim to offer another view of this problem in a
perspective assuming the reason behind this accuracy drop is the reliance of
models on the features that are not aligned well with how a data annotator
considers similar across these two datasets. We refer to these features as
misaligned features. We extend the conventional generalization error bound to a
new one for this setup with the knowledge of how the misaligned features are
associated with the label. Our analysis offers a set of techniques for this
problem, and these techniques are naturally linked to many previous methods in
robust machine learning literature. We also compared the empirical strength of
these methods demonstrated the performance when these previous techniques are
combined, with an implementation available at https://github.com/OoDBag/WR
