  Practical natural language processing (NLP) tasks are commonly long-tailed
with noisy labels. Those problems challenge the generalization and robustness
of complex models such as Deep Neural Networks (DNNs). Some commonly used
resampling techniques, such as oversampling or undersampling, could easily lead
to overfitting. It is growing popular to learn the data weights leveraging a
small amount of metadata. Besides, recent studies have shown the advantages of
self-supervised pre-training, particularly to the under-represented data. In
this work, we propose a general framework to handle the problem of both
long-tail and noisy labels. The model is adapted to the domain of problems in a
contrastive learning manner. The re-weighting module is a feed-forward network
that learns explicit weighting functions and adapts weights according to
metadata. The framework further adapts weights of terms in the loss function
through a combination of the polynomial expansion of cross-entropy loss and
focal loss. Our extensive experiments show that the proposed framework
consistently outperforms baseline methods. Lastly, our sensitive analysis
emphasizes the capability of the proposed framework to handle the long-tailed
problem and mitigate the negative impact of noisy labels.
