  In this paper, we investigate the effects of releasing arXiv preprints of
papers that are undergoing a double-blind review process. In particular, we ask
the following research question: What is the relation between de-anonymization
of authors through arXiv preprints and acceptance of a research paper at a
(nominally) double-blind venue? Under two conditions: papers that are released
on arXiv before the review phase and papers that are not, we examine the
correlation between the reputation of their authors with the review scores and
acceptance decisions. By analyzing a dataset of ICLR 2020 and ICLR 2019
submissions (n=5050), we find statistically significant evidence of positive
correlation between percentage acceptance and papers with high reputation
released on arXiv. In order to understand this observed association better, we
perform additional analyses based on self-specified confidence scores of
reviewers and observe that less confident reviewers are more likely to assign
high review scores to papers with well known authors and low review scores to
papers with less known authors, where reputation is quantified in terms of
number of Google Scholar citations. We emphasize upfront that our results are
purely correlational and we neither can nor intend to make any causal claims. A
blog post accompanying the paper and our scraping code will be linked in the
project website https://sites.google.com/view/deanon-arxiv/home
