  In this paper, we propose sentiment classification models based on BERT
integrated with DRO (Distributionally Robust Classifiers) to improve model
performance on datasets with distributional shifts. We added 2-Layer Bi-LSTM,
projection layer (onto simplex or Lp ball), and linear layer on top of BERT to
achieve distributionally robustness. We considered one form of distributional
shift (from IMDb dataset to Rotten Tomatoes dataset). We have confirmed through
experiments that our DRO model does improve performance on our test set with
distributional shift from the training set.
