  From CNNs to attention mechanisms, encoding inductive biases into neural
networks has been a fruitful source of improvement in machine learning. Adding
auxiliary losses to the main objective function is a general way of encoding
biases that can help networks learn better representations. However, since
auxiliary losses are minimized only on training data, they suffer from the same
generalization gap as regular task losses. Moreover, by adding a term to the
loss function, the model optimizes a different objective than the one we care
about. In this work we address both problems: first, we take inspiration from
\textit{transductive learning} and note that after receiving an input but
before making a prediction, we can fine-tune our networks on any unsupervised
loss. We call this process {\em tailoring}, because we customize the model to
each input to ensure our prediction satisfies the inductive bias. Second, we
formulate {\em meta-tailoring}, a nested optimization similar to that in
meta-learning, and train our models to perform well on the task objective after
adapting them using an unsupervised loss. The advantages of tailoring and
meta-tailoring are discussed theoretically and demonstrated empirically on a
diverse set of examples.
