<p>Anonymous account for obvious reasons. I am a researcher in TCS. I have several publications in SODA/STOC/FOCS. I've recently been so disgruntled with the way these conferences are run, and wanted to voice my frustration into the void. It seems like the organizers collect &quot;feedback&quot; every year, and throw it into the bin. In particular,</p>
<ol>
<li><p>In what universe does the single chair model still make sense? How is the chair able to supervise the reviewing process for all PCs acurately? Every single discipline I know has switched to having several area chairs for different subdisciplines. Why are theory conferences lagging?</p>
</li>
<li><p>What steps are being taken to prevent crap reviews? I've seen one of my students' SODA 2024 reviews, and they were a hot steaming pile of shit. The reviewers had &quot;Expert&quot; for confidence, when it was clear as day light the reviewers (external or PCs) had no idea what the paper is talking about. Some reviews were 2-3 lines, and I have no idea how that wasn't caught by a PC. What steps are being taken by the community to weed out low quality reviews and reviewers? Has <em>any</em> reviewer been banned from reviewing in (say) the last 5 years for the quality of their reviews?</p>
</li>
<li><p>I am aware of a PC who has served several times on the committee in the last 9 years who notoriously reads the paper in their pile and writes reviews on the last day. Their reviews are notoriously utter garbage, and they get into flaming wars arguing for their BS reviews. I know they've sank several papers that deserved to get in. Why is this PC still being invited to serve on the commitee when they have shown such utter disregard to the seriousness of their task in objectively reviewing papers?</p>
</li>
<li><p>I was optimistic when conferences started implementing double blind reviews, but I am now convinced that's complete and utter BS. I've recieved emails by known researchers in my area who publicized their paper after the SODA rebuttals period saying &quot;they hoped to see it in SODA despite the poor scores&quot;. These papers, surprise surprise, got into this and last year's SODA. Not to mention a few bad actors that I am familiar with who search the name of the paper they are asked to review on arxiv, despite the instructions specifically asking not to do that. Do the conferences actually take double blind reviews seriously, or is this just lip service?</p>
</li>
<li><p>Who is doing quality assurance on the papers that get in? There are for example 2 papers in last year's FOCS by 2 &quot;superstars&quot; where I was able to prove the statements in an hour. These are papers that claimed their main contribution was a &quot;non-trivial&quot; proof of such statements. Meanwhile, the papers I root for get weeded out and have to go through several rejection cycles.</p>
</li>
<li><p>Why are poster sessions not a thing yet? Conferences complain that they can &quot;only accept so many papers&quot; because of physical constraints, but I believe this is utter BS. ML/Data mining conferences accept &quot;good&quot; papers, and categorize them as posters, oral, and spotlight papers. Poster papers are presented during a poster session, whereas the oral/spotlight papers get to present.</p>
</li>
<li><p>What is with the obsession with acceptance rates? This is just driving people away from TCS. The theory community keeps blabbering on wanting to be more inclusive to new researchers, but I know of at least 3 researches (including 2 of my own students) who are so sick of the BS happening in reviewing, and waiting several cycles of rejections for a paper to get in, that they switched to publishing almost solely in ML conferences. These same papers are still being cited by SODA/STOC/FOCS papers and are clearly relevant to the TCS community. Can we drop the facade of &quot;acceptance rates&quot; and just focus on accepting solid good papers?</p>
</li>
</ol>
<p>As I near the end of my career, I'm filled with nothing but frustration with the way that TCS conferences are run. I actively encourage all my students to switch fields, including the really talented ones. Every time I discuss this with a TCS researcher, they share my frustration, but argue that &quot;this is how it's always been&quot;.</p>

CC BY-SA 4.0
2023-11-11T05:32:33.357Z
About acceptance rates, they are much higher for top theory conferences than for top conferences in almost any other CS subfield. Also, coming from publishing in AAAI/IJCAI during my masters to publishing in theory in my phd, theory definitely compares positively on the quality of reviews and in the seriousness of the reviewing process in my experience.
Laakeri
CC BY-SA 4.0
2023-11-11T05:36:38.877Z
@Laakeri that's not true. NeurIPS acceptance rates for 2022 was 25.6%, and 26.1% for 2023. Sigmod (for datamining) has an acceptance rate of 29.3% in 2022, and 41.8% in 2021 (I can't find figures for 2023). Compare this to for example SODA (28.1%, 30%), STOC (25.7%, 26.9%) and FOCS (37%, 34%).
DisgruntledTCSResearcher
CC BY-SA 4.0
2023-11-11T05:39:56.470Z
@Laakeri Also, I don't think it's constructive to compare acceptance rates across disciplines. The average ML submission is not the same as the average TCS submission. There is a ton of self-selection that happens for TCS. The current enforced acceptance rates are causing extremely solid and good papers to be rejected, when that should not be happening.
DisgruntledTCSResearcher
CC BY-SA 4.0
2023-11-11T06:53:46.370Z
Agree with the fact that acceptance rates are not comparable because of self selection. I think I misunderstood your point 7 a bit.
Laakeri
CC BY-SA 4.0
2023-11-11T08:00:53.003Z
As for crap reviews, this is inherent in the idea of the conference publication model. There is no consistent way to get quality reviews on a ridiculously short deadline such as a few weeks. Theoretical computer scientists should abandon this model and start publishing in journals, as all other scientists are doing.
Emil Jeřábek
CC BY-SA 4.0
2023-11-11T08:54:07.143Z
@DisgruntledTCSResearcher you state that we shouldn't compare acceptance rates between disciplines, but several of your points keep comparing TCS and ML conferences. I'm not saying all is rosy in TCS (it's not), but I haven't met a single researcher in ML (theory or not) who doesn't say the ML conference reviewing process is utterly broken.
Clement C.
CC BY-SA 4.0
2023-11-11T17:22:21.957Z
I would point out that there are many TCS researchers who also try to publish in ML venues. There the work can be somewhere between theory and practice. While these conferences have another set of issues, at least they are quite open to papers relating to both theory and experiments. At least for me, they are a much more appealing venue than STOC/FOCS/SODA.
Sandeep Silwal
CC BY-SA 4.0
2023-11-17T13:46:10.497Z
Is this a Theory A vs. Theory B issue? I have severed on a bunch of PCs covering topics such as logic in computer science, semantics of programming languages, type theory, category-theoretic aspects of TCS, etc. (LICS, ICFP, POPL, LCS, TACL). I have seem some bad reviewes, but nowhere as bad as what you are describing, and for the most part reviewers do as good as job as they can be expected to. PC chairs seem to have a good handle on things, too. (Acceptance rates and conferences-as-journals is another matter.)
Andrej Bauer
CC BY-SA 4.0
2023-11-17T13:48:28.097Z
BTW, SIGPLAN conferences (ICFP, OOPSLA, POPL) have moved to a model where the publication undergoes a journal-style review process, and the proceedings want to be a journal (https://dl.acm.org/journal/pacmpl).
Andrej Bauer
CC BY-SA 4.0
2023-11-17T21:24:59.303Z
I would be so happy to be banned from refereeing for computer science conferences ...
user313032
CC BY-SA 4.0
2023-11-17T22:17:06.770Z
As much as I agree with many of the points in the question and many other things that are broken with the academic system (cf https://a3nm.net/work/research/wrong/), I'm quite surprised with point 5. In my experience, accepted papers at major conferences can have many problems but I have never seen results that were as trivial as you described. Since you are anonymous anyway, could you point us to these specific FOCS papers? I think this would be a compelling argument against the perceived prestige of these conferences.
Antoine Amarilli 'a3nm'
CC BY-SA 4.0
2023-11-21T05:59:54.597Z
As someone who usually publishes on AAAI/IJCAI, I can say the trash reviews are very common there as well. However, TCS conferences are smaller so I don’t see excuses regarding the inability of checking for obviously bad reviews.
Nicola Gigante
CC BY-SA 4.0
2024-06-16T21:24:44.163Z
In regards to point 7) Unless you believe every paper has the same value you have to accept that the concept of citing a paper that has true and useful results but just doesn't have as much novel nontrivial material.  Have you considered the low acceptance rate is due to many people wanting their work in high tier journals? Thats not to say that it's always done objectively and correctly.
Hao S
CC BY-SA 4.0
2024-07-02T18:48:28.567Z
I think even though it is "theoretical" computer science research, most of the papers are about some hard combinatorial problem solving rather than theory building. Difficult computations are appreciated by the reviewers; if your paper do not fit this mold then getting papers accepted is really hard nowadays.
A.2
CC BY-SA 4.0
2024-10-16T16:39:44.163Z
This reads more like a complaint than a question. I will likely close it unless convinced otherwise.
Lev Reyzin
<conferences>
Are theoretical computer science conferences losing touch with reality?
DisgruntledTCSResearcher
# Are theoretical computer science conferences losing touch with reality?

(Asked by: username_0 on 2023-11-11)

Anonymous account for obvious reasons. I am a researcher in TCS. I have several publications in SODA/STOC/FOCS. I've recently been so disgruntled with the way these conferences are run, and wanted to voice my frustration into the void. It seems like the organizers collect "feedback" every year, and throw it into the bin. In particular,

1. In what universe does the single chair model still make sense? How is the chair able to supervise the reviewing process for all PCs acurately? Every single discipline I know has switched to having several area chairs for different subdisciplines. Why are theory conferences lagging?
2. What steps are being taken to prevent crap reviews? I've seen one of my students' SODA 2024 reviews, and they were a hot steaming pile of shit. The reviewers had "Expert" for confidence, when it was clear as day light the reviewers (external or PCs) had no idea what the paper is talking about. Some reviews were 2-3 lines, and I have no idea how that wasn't caught by a PC. What steps are being taken by the community to weed out low quality reviews and reviewers? Has *any* reviewer been banned from reviewing in (say) the last 5 years for the quality of their reviews?
3. I am aware of a PC who has served several times on the committee in the last 9 years who notoriously reads the paper in their pile and writes reviews on the last day. Their reviews are notoriously utter garbage, and they get into flaming wars arguing for their BS reviews. I know they've sank several papers that deserved to get in. Why is this PC still being invited to serve on the commitee when they have shown such utter disregard to the seriousness of their task in objectively reviewing papers?
4. I was optimistic when conferences started implementing double blind reviews, but I am now convinced that's complete and utter BS. I've recieved emails by known researchers in my area who publicized their paper after the SODA rebuttals period saying "they hoped to see it in SODA despite the poor scores". These papers, surprise surprise, got into this and last year's SODA. Not to mention a few bad actors that I am familiar with who search the name of the paper they are asked to review on arxiv, despite the instructions specifically asking not to do that. Do the conferences actually take double blind reviews seriously, or is this just lip service?
5. Who is doing quality assurance on the papers that get in? There are for example 2 papers in last year's FOCS by 2 "superstars" where I was able to prove the statements in an hour. These are papers that claimed their main contribution was a "non-trivial" proof of such statements. Meanwhile, the papers I root for get weeded out and have to go through several rejection cycles.
6. Why are poster sessions not a thing yet? Conferences complain that they can "only accept so many papers" because of physical constraints, but I believe this is utter BS. ML/Data mining conferences accept "good" papers, and categorize them as posters, oral, and spotlight papers. Poster papers are presented during a poster session, whereas the oral/spotlight papers get to present.
7. What is with the obsession with acceptance rates? This is just driving people away from TCS. The theory community keeps blabbering on wanting to be more inclusive to new researchers, but I know of at least 3 researches (including 2 of my own students) who are so sick of the BS happening in reviewing, and waiting several cycles of rejections for a paper to get in, that they switched to publishing almost solely in ML conferences. These same papers are still being cited by SODA/STOC/FOCS papers and are clearly relevant to the TCS community. Can we drop the facade of "acceptance rates" and just focus on accepting solid good papers?

As I near the end of my career, I'm filled with nothing but frustration with the way that TCS conferences are run. I actively encourage all my students to switch fields, including the really talented ones. Every time I discuss this with a TCS researcher, they share my frustration, but argue that "this is how it's always been".

**Tags:** conferences

> **Comments:**
> **username_1**: About acceptance rates, they are much higher for top theory conferences than for top conferences in almost any other CS subfield. Also, coming from publishing in AAAI/IJCAI during my masters to publishing in theory in my phd, theory definitely compares positively on the quality of reviews and in the seriousness of the reviewing process in my experience.  
> **username_0**: @username_1 that's not true. NeurIPS acceptance rates for 2022 was 25.6%, and 26.1% for 2023. Sigmod (for datamining) has an acceptance rate of 29.3% in 2022, and 41.8% in 2021 (I can't find figures for 2023). Compare this to for example SODA (28.1%, 30%), STOC (25.7%, 26.9%) and FOCS (37%, 34%).  
> **username_0**: @username_1 Also, I don't think it's constructive to compare acceptance rates across disciplines. The average ML submission is not the same as the average TCS submission. There is a ton of self-selection that happens for TCS. The current enforced acceptance rates are causing extremely solid and good papers to be rejected, when that should not be happening.  
> **username_1**: Agree with the fact that acceptance rates are not comparable because of self selection. I think I misunderstood your point 7 a bit.  
> **username_2**: As for crap reviews, this is inherent in the idea of the conference publication model. There is no consistent way to get quality reviews on a ridiculously short deadline such as a few weeks. Theoretical computer scientists should abandon this model and start publishing in journals, as all other scientists are doing.  
> **username_3**: @username_0 you state that we shouldn't compare acceptance rates between disciplines, but several of your points keep comparing TCS and ML conferences. I'm not saying all is rosy in TCS (it's not), but I haven't met a single researcher in ML (theory or not) who doesn't say the ML conference reviewing process is utterly broken.  
> **username_4**: I would point out that there are many TCS researchers who also try to publish in ML venues. There the work can be somewhere between theory and practice. While these conferences have another set of issues, at least they are quite open to papers relating to both theory and experiments. At least for me, they are a much more appealing venue than STOC/FOCS/SODA.  
> **username_5**: Is this a Theory A vs. Theory B issue? I have severed on a bunch of PCs covering topics such as logic in computer science, semantics of programming languages, type theory, category-theoretic aspects of TCS, etc. (LICS, ICFP, POPL, LCS, TACL). I have seem some bad reviewes, but nowhere as bad as what you are describing, and for the most part reviewers do as good as job as they can be expected to. PC chairs seem to have a good handle on things, too. (Acceptance rates and conferences-as-journals is another matter.)  
> **username_5**: BTW, SIGPLAN conferences (ICFP, OOPSLA, POPL) have moved to a model where the publication undergoes a journal-style review process, and the proceedings want to be a journal (https://dl.acm.org/journal/pacmpl).  
> **username_6**: I would be so happy to be banned from refereeing for computer science conferences ...  
> **username_7**: As much as I agree with many of the points in the question and many other things that are broken with the academic system (cf https://a3nm.net/work/research/wrong/), I'm quite surprised with point 5. In my experience, accepted papers at major conferences can have many problems but I have never seen results that were as trivial as you described. Since you are anonymous anyway, could you point us to these specific FOCS papers? I think this would be a compelling argument against the perceived prestige of these conferences.  
> **username_8**: As someone who usually publishes on AAAI/IJCAI, I can say the trash reviews are very common there as well. However, TCS conferences are smaller so I don’t see excuses regarding the inability of checking for obviously bad reviews.  
> **username_9**: In regards to point 7) Unless you believe every paper has the same value you have to accept that the concept of citing a paper that has true and useful results but just doesn't have as much novel nontrivial material.  Have you considered the low acceptance rate is due to many people wanting their work in high tier journals? Thats not to say that it's always done objectively and correctly.  
> **username_10**: I think even though it is "theoretical" computer science research, most of the papers are about some hard combinatorial problem solving rather than theory building. Difficult computations are appreciated by the reviewers; if your paper do not fit this mold then getting papers accepted is really hard nowadays.  
> **username_11**: This reads more like a complaint than a question. I will likely close it unless convinced otherwise.  

---

## 0 answers

