<p>This is not an answer to the question but too long for a comment by 1253 characters.</p>
<p>STACS 2021 used light double blind review. We used easychair for it, which provides some functionality for double blind review. You need the executive licence, which costs some money. Costs depend on the number of submissions, I think for STACS it was around 600 British pounds.</p>
<p>Overall, the process went quite smoothly. As a PC co-chair, you can of course see the authors. So I actively only handled very few papers, mostly those that no one wanted to have and there, I was rather far away from the topic. Instead I focused more on monitoring the process, stimulating discussion etc.</p>
<p>Authors can declare conflicts of interests when submitting. I had the feeling that this was used in an honest way. Furthermore, PC members get to see a list of all authors (without the paper titles) and can declare conflicts of interest against authors. This eliminates essentially all conflicts with PC members. (As a PC chair, you can see the
list of all COIs. The authors' declarations and the PC's declarations matched with very little noise.)</p>
<p>More problematic are the subreferees. Easychair blocks referee request to authors of a paper automatically (if you use the email addresses from the list provided by easychair). Subreferees were asked to declare COIs if they were aware of it. In most cases, this worked well. There were a few cases (maybe 5 out of &gt; 600) were I had the feeling that the subreferee correctly guessed the authors and wrote a biased (very positive) report. One could have a person (or committee) here, who sole purpose is to look for such conflicts but who is not involved in the PC work.</p>
<p>In a questionaire fill out after the notification, about 85% of the subreferees said that they did not actively searched for the authors. On the other hand, about 50% were able to guess the authors of at least some of  papers they reviewed.</p>

CC BY-SA 4.0
2021-06-14T13:36:17.947Z
2021-06-14T13:36:17.947Z
Markus Bläser
<p>As of 2023, the situation has changed, and all conferences mentioned in the question (and, indeed, all major generalistic conferences in TCS that I am aware of) are now using double-blind reviewing:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Symposium_on_Theoretical_Aspects_of_Computer_Science" rel="noreferrer">STACS</a> since <a href="http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=110612&amp;copyownerid=164547" rel="noreferrer">2021</a>,</li>
<li><a href="https://en.wikipedia.org/wiki/International_Colloquium_on_Automata,_Languages_and_Programming" rel="noreferrer">ICALP</a> since <a href="http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=152726&amp;copyownerid=172199" rel="noreferrer">2022</a>,</li>
<li><a href="https://en.wikipedia.org/wiki/Symposium_on_Foundations_of_Computer_Science" rel="noreferrer">FOCS</a> since <a href="https://focs2022.eecs.berkeley.edu/cfp.html" rel="noreferrer">2022</a>,</li>
<li><a href="https://en.wikipedia.org/wiki/Symposium_on_Theory_of_Computing" rel="noreferrer">STOC</a> since <a href="http://acm-stoc.org/stoc2024/" rel="noreferrer">2024</a>,</li>
<li><a href="https://en.wikipedia.org/wiki/Symposium_on_Discrete_Algorithms" rel="noreferrer">SODA</a> since <a href="https://www.siam.org/conferences/cm/submissions-and-deadlines/soda22-submissions-deadlines" rel="noreferrer">2022</a>,</li>
<li><a href="https://en.wikipedia.org/wiki/European_Symposium_on_Algorithms" rel="noreferrer">ESA</a> since
<a href="http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=87922&amp;copyownerid=35838" rel="noreferrer">2019</a>.</li>
</ul>
<p>See also the website by @EmeryBerger tracking this information: <a href="https://double-blind.org" rel="noreferrer">double-blind.org</a>.</p>
<p>However, as far as I know, the same is not true of TCS journals.</p>

CC BY-SA 4.0
2023-09-17T02:58:06.207Z
Not that it has helped overcome the cons of being not double-blind...
user3508551
CC BY-SA 4.0
2023-09-16T22:01:46.413Z
2023-09-16T22:11:42.157Z
2023-09-16T22:11:42.157Z
Antoine Amarilli 'a3nm'
<p>I have found that the most notable TCS conferences like <a href="https://focs2021.cs.colorado.edu/cfp/" rel="noreferrer">FOCS</a>, <a href="http://acm-stoc.org/stoc2021/stoc-cfp.html" rel="noreferrer">STOC</a>, <a href="https://www.siam.org/conferences/cm/submissions-and-deadlines/soda21-submissions-deadlines" rel="noreferrer">SODA</a>, and <a href="http://easyconferences.eu/icalp2021/cfp/" rel="noreferrer">ICALP</a> are single-blind. That is the authors do not know the identity of reviewers; however, the reviewers know the identity of the authors.</p>
<p>I believe that the most ethical way of reviewing is the double-blind review, where neither the authors nor reviewers know the identity of each other.
Then, why the top TCS conferences are just single-blind? What are the good reasons for doing it?</p>
<p>The only reason that I could think of is that a reputed author's name or institution could boost the confidence of the reviewer about the claims made in the paper. But again it can introduce bias.</p>
<hr />
<p>Note: There are few conferences that do follow double-blind reviewing like <a href="http://algo2021.tecnico.ulisboa.pt/ESA2021/" rel="noreferrer">ESA</a>. Then, why this practice is not followed by the other top conferences?</p>

CC BY-SA 4.0
2021-06-08T13:15:39.667Z
SoCG is currently having a vote centred around this question. Will be interesting to see the results and what the CG community thinks about that.
Tassle
CC BY-SA 4.0
2021-06-09T04:33:58.517Z
Also there is a problem with posting your paper on arxiv when the reviewing process is double blind.
Bartosz Bednarczyk
CC BY-SA 4.0
2021-06-09T13:37:04.717Z
None of these arguments against double-blind work if we do not require the process to be perfect, but just to aim to reduce the initial bias. I think there are no downsides in double-blind if it amounts to just omitting names and affiliations in the submission to the conference server. This is how the lightweight double-blind works in e.g. ESA, STACS, LICS, and PODC.
Laakeri
CC BY-SA 4.0
2021-06-09T17:02:35.127Z
@Laakeri: I completely agree and would upvote this 100x if I could (the phrase I've been using is "don't let the perfect be the enemy of the good"). The main issue is then logistics, especially around handling conflicts of interest while maintaining double-blind. But I have found out from some organizers who have done this lightweight double-blind that there is a fairly easy way to handle this in easychair. I hope more conferences do this in the future.
Joshua Grochow
CC BY-SA 4.0
2021-06-10T00:34:19.087Z
CCC is still not double-anon either. (Re my previous comment: I've asked one of the organizers I spoke to if they would feel comfortable posting here.)
Joshua Grochow
CC BY-SA 4.0
2021-06-11T16:53:26.600Z
STOC is supposedly going double blind (forced to by ACM).
Yuval Filmus
CC BY-SA 4.0
2021-06-11T16:53:48.373Z
Some AI conferences are now going triple blind: PC members don't know who the reviewers are.
Yuval Filmus
CC BY-SA 4.0
2021-06-11T16:56:23.253Z
You mention that reputation can help a paper. Similarly, bad reputation can work against a paper. If researcher X is known to have errors in their papers, the PC might review their papers more carefully.
Yuval Filmus
CC BY-SA 4.0
2021-06-12T15:32:23.290Z
There is an unstated assumption here that not only does OP think that double-blind is best practice, but in fact double-blind is objectively the best, and so the real question here, is why have TCS conferences "not seen the light". I think that a better way of stating the question would be, why did most TCS conferences not switch to double-blind, although many other CS conferences have.
Yuval Filmus
CC BY-SA 4.0
2021-06-12T15:34:24.197Z
One could ask, why are reviews in TCS conferences not made public? I believe that this is the most ethical choice, and the only reason I can think of for TCS conferences not switching, is that reviewers are less likely to be honest (but this is offset by some other, unstated advantages, that are obvious).
Yuval Filmus
CC BY-SA 4.0
2021-06-14T13:56:33.523Z
FWIW SODA this year (SODA 2022) is doing a "lightweight double-blind reviewing process" (https://www.siam.org/conferences/cm/submissions-and-deadlines/soda22-submissions-deadlines).
Neal Young
CC BY-SA 4.0
2021-06-14T15:35:58.603Z
@YuvalFilmus: Public review solves some problems but not others. For example, public review would allow *post hoc* checking of bias, whereas double-anon is supposed to reduce bias *before* decisions are made. Now that some TCS conferences are starting to do double-anon, it would be great to see some data on the outcomes, e.g. measures of bias before vs after going double-anon, over several years.
Joshua Grochow
CC BY-SA 4.0
2021-06-14T15:40:09.223Z
To the people who voted to close b/c answers would be opinion-based: I disagree. Q1 is about why conferences aren't yet double-anon. This could, in principle, be answered factually by people involved in organizing those confs. Q2 is about "good reasons for doing it". While *theorems* in this area are impossible - it's about the science of people - there can be good reasons that are as un-opinion-based as any when it comes to the science of people.
Joshua Grochow
CC BY-SA 4.0
2021-06-15T02:57:01.413Z
I think it is a mistake NOT to have a double-blind review for the CS theory conferences. There has to be a rebuttal process as well because some reviewers write whatever nonsense and baseless comments they like.
Arash Rafiey
CC BY-SA 4.0
2021-06-17T17:12:34.303Z
Also FWIW this site (which I curate) tracks double-blind reviewing status of top conferences across areas of computer science: https://double-blind.org. In addition, it contains links pointing to resources related to double-blind reviewing (academic papers, articles, and blog posts).
EmeryBerger
CC BY-SA 4.0
2023-09-16T22:03:44.107Z
@YuvalFilmus: would you happen to have a reference about how ACM imposed a double-blind policy on STOC? I was not aware of ACM imposing this kind of choices on member conferences.
Antoine Amarilli 'a3nm'
CC BY-SA 4.0
2023-10-29T13:50:08.633Z
@a3nm If I remember correctly, it was mentioned on Slack or equivalent, which wouldn't necessarily count as a reference.
Yuval Filmus
<soft-question>
Why most of the top TCS conferences are not double-blind?
Inuyasha Yagami
# Why most of the top TCS conferences are not double-blind?

(Asked by: username_0 on 2021-06-08)

I have found that the most notable TCS conferences like [FOCS](https://focs2021.cs.colorado.edu/cfp/), [STOC](http://acm-stoc.org/stoc2021/stoc-cfp.html), [SODA](https://www.siam.org/conferences/cm/submissions-and-deadlines/soda21-submissions-deadlines), and [ICALP](http://easyconferences.eu/icalp2021/cfp/) are single-blind. That is the authors do not know the identity of reviewers; however, the reviewers know the identity of the authors.

I believe that the most ethical way of reviewing is the double-blind review, where neither the authors nor reviewers know the identity of each other.
Then, why the top TCS conferences are just single-blind? What are the good reasons for doing it?

The only reason that I could think of is that a reputed author's name or institution could boost the confidence of the reviewer about the claims made in the paper. But again it can introduce bias.

---

Note: There are few conferences that do follow double-blind reviewing like [ESA](http://algo2021.tecnico.ulisboa.pt/ESA2021/). Then, why this practice is not followed by the other top conferences?

**Tags:** soft-question

> **Comments:**
> **username_1**: SoCG is currently having a vote centred around this question. Will be interesting to see the results and what the CG community thinks about that.  
> **username_2**: Also there is a problem with posting your paper on arxiv when the reviewing process is double blind.  
> **username_3**: None of these arguments against double-blind work if we do not require the process to be perfect, but just to aim to reduce the initial bias. I think there are no downsides in double-blind if it amounts to just omitting names and affiliations in the submission to the conference server. This is how the lightweight double-blind works in e.g. ESA, STACS, LICS, and PODC.  
> **username_4**: @username_3: I completely agree and would upvote this 100x if I could (the phrase I've been using is "don't let the perfect be the enemy of the good"). The main issue is then logistics, especially around handling conflicts of interest while maintaining double-blind. But I have found out from some organizers who have done this lightweight double-blind that there is a fairly easy way to handle this in easychair. I hope more conferences do this in the future.  
> **username_4**: CCC is still not double-anon either. (Re my previous comment: I've asked one of the organizers I spoke to if they would feel comfortable posting here.)  
> **username_5**: STOC is supposedly going double blind (forced to by ACM).  
> **username_5**: Some AI conferences are now going triple blind: PC members don't know who the reviewers are.  
> **username_5**: You mention that reputation can help a paper. Similarly, bad reputation can work against a paper. If researcher X is known to have errors in their papers, the PC might review their papers more carefully.  
> **username_5**: There is an unstated assumption here that not only does OP think that double-blind is best practice, but in fact double-blind is objectively the best, and so the real question here, is why have TCS conferences "not seen the light". I think that a better way of stating the question would be, why did most TCS conferences not switch to double-blind, although many other CS conferences have.  
> **username_5**: One could ask, why are reviews in TCS conferences not made public? I believe that this is the most ethical choice, and the only reason I can think of for TCS conferences not switching, is that reviewers are less likely to be honest (but this is offset by some other, unstated advantages, that are obvious).  
> **username_6**: FWIW SODA this year (SODA 2022) is doing a "lightweight double-blind reviewing process" (https://www.siam.org/conferences/cm/submissions-and-deadlines/soda22-submissions-deadlines).  
> **username_4**: @YuvalFilmus: Public review solves some problems but not others. For example, public review would allow *post hoc* checking of bias, whereas double-anon is supposed to reduce bias *before* decisions are made. Now that some TCS conferences are starting to do double-anon, it would be great to see some data on the outcomes, e.g. measures of bias before vs after going double-anon, over several years.  
> **username_4**: To the people who voted to close b/c answers would be opinion-based: I disagree. Q1 is about why conferences aren't yet double-anon. This could, in principle, be answered factually by people involved in organizing those confs. Q2 is about "good reasons for doing it". While *theorems* in this area are impossible - it's about the science of people - there can be good reasons that are as un-opinion-based as any when it comes to the science of people.  
> **username_7**: I think it is a mistake NOT to have a double-blind review for the CS theory conferences. There has to be a rebuttal process as well because some reviewers write whatever nonsense and baseless comments they like.  
> **username_8**: Also FWIW this site (which I curate) tracks double-blind reviewing status of top conferences across areas of computer science: https://double-blind.org. In addition, it contains links pointing to resources related to double-blind reviewing (academic papers, articles, and blog posts).  
> **username_9**: @YuvalFilmus: would you happen to have a reference about how ACM imposed a double-blind policy on STOC? I was not aware of ACM imposing this kind of choices on member conferences.  
> **username_5**: @a3nm If I remember correctly, it was mentioned on Slack or equivalent, which wouldn't necessarily count as a reference.  

---

## 2 answers

### Answer by username_10 on 2021-06-14. Score: 16

This is not an answer to the question but too long for a comment by 1253 characters.

STACS 2021 used light double blind review. We used easychair for it, which provides some functionality for double blind review. You need the executive licence, which costs some money. Costs depend on the number of submissions, I think for STACS it was around 600 British pounds.

Overall, the process went quite smoothly. As a PC co-chair, you can of course see the authors. So I actively only handled very few papers, mostly those that no one wanted to have and there, I was rather far away from the topic. Instead I focused more on monitoring the process, stimulating discussion etc.

Authors can declare conflicts of interests when submitting. I had the feeling that this was used in an honest way. Furthermore, PC members get to see a list of all authors (without the paper titles) and can declare conflicts of interest against authors. This eliminates essentially all conflicts with PC members. (As a PC chair, you can see the
list of all COIs. The authors' declarations and the PC's declarations matched with very little noise.)

More problematic are the subreferees. Easychair blocks referee request to authors of a paper automatically (if you use the email addresses from the list provided by easychair). Subreferees were asked to declare COIs if they were aware of it. In most cases, this worked well. There were a few cases (maybe 5 out of > 600) were I had the feeling that the subreferee correctly guessed the authors and wrote a biased (very positive) report. One could have a person (or committee) here, who sole purpose is to look for such conflicts but who is not involved in the PC work.

In a questionaire fill out after the notification, about 85% of the subreferees said that they did not actively searched for the authors. On the other hand, about 50% were able to guess the authors of at least some of papers they reviewed.


---

### Answer by username_9 on 2023-09-16. Score: 12

As of 2023, the situation has changed, and all conferences mentioned in the question (and, indeed, all major generalistic conferences in TCS that I am aware of) are now using double-blind reviewing:

* [STACS](https://en.wikipedia.org/wiki/Symposium_on_Theoretical_Aspects_of_Computer_Science) since [2021](http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=110612©ownerid=164547),
* [ICALP](https://en.wikipedia.org/wiki/International_Colloquium_on_Automata,_Languages_and_Programming) since [2022](http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=152726©ownerid=172199),
* [FOCS](https://en.wikipedia.org/wiki/Symposium_on_Foundations_of_Computer_Science) since [2022](https://focs2022.eecs.berkeley.edu/cfp.html),
* [STOC](https://en.wikipedia.org/wiki/Symposium_on_Theory_of_Computing) since [2024](http://acm-stoc.org/stoc2024/),
* [SODA](https://en.wikipedia.org/wiki/Symposium_on_Discrete_Algorithms) since [2022](https://www.siam.org/conferences/cm/submissions-and-deadlines/soda22-submissions-deadlines),
* [ESA](https://en.wikipedia.org/wiki/European_Symposium_on_Algorithms) since
  [2019](http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=87922©ownerid=35838).

See also the website by @username_8 tracking this information: [double-blind.org](https://double-blind.org).

However, as far as I know, the same is not true of TCS journals.

> **Comments:**
> **username_11**: Not that it has helped overcome the cons of being not double-blind...  

---

