  Two step document ranking, where the initial retrieval is done by a classical
information retrieval method, followed by neural re-ranking model, is the new
standard. The best performance is achieved by using transformer-based models as
re-rankers, e.g., BERT. We employ Longformer, a BERT-like model for long
documents, on the MS MARCO document re-ranking task. The complete code used for
training the model can be found on:
https://github.com/isekulic/longformer-marco
