  Double-blind conferences have engaged in debates over whether to allow
authors to post their papers online on arXiv or elsewhere during the review
process. Independently, some authors of research papers face the dilemma of
whether to put their papers on arXiv due to its pros and cons. We conduct a
study to substantiate this debate and dilemma via quantitative measurements.
Specifically, we conducted surveys of reviewers in two top-tier double-blind
computer science conferences -- ICML 2021 (5361 submissions and 4699 reviewers)
and EC 2021 (498 submissions and 190 reviewers). Our two main findings are as
follows. First, more than a third of the reviewers self-report searching online
for a paper they are assigned to review. Second, outside the review process, we
find that preprints from better-ranked affiliations see a weakly higher
visibility, with a correlation of 0.06 in ICML and 0.05 in EC. In particular,
papers associated with the top-10-ranked affiliations had a visibility of
approximately 11% in ICML and 22% in EC, whereas the remaining papers had a
visibility of 7% and 18% respectively.
