<blockquote>
  <p>How does a junior researcher write a negative review?</p>
</blockquote>

<p>By writing a long, thorough, honest, but polite review.</p>

<p>It helps if you address the <em>paper</em>, not the <em>authors</em>. Make it as non-personal as possible. Say how the <em>paper</em> should be revised, not what the <em>authors</em> should do. Say that Theorem 3 is wrong, not that the authors are wrong.</p>

<blockquote>
  <p>What should you do if you have specific technical comments/improvements that are too long for a standard review?</p>
</blockquote>

<p>There is no such thing as "too long for a review".</p>

<p>Sure, senior researchers write one-line reviews, but that does not mean that a four-page review is too long for a conference paper. If you write a long review, of course anyone will immediately guess that this is written by a junior researcher, but I do not see any reason to avoid it.</p>

<hr>

<p>However, if you write a long review, it is extremely important that it is well-structured and that anyone can find your <em>recommendation</em> (i.e., to publish or not, and very briefly why) in two seconds. Make your recommendation stand out, and make sure this part (usually very near the beginning of your review) is very brief.</p>

CC BY-SA 2.5
2011-02-22T11:12:11.120Z
Start your review by your recommendation : very good advice !
Sylvain Peyronnet
CC BY-SA 2.5
2011-02-22T12:02:05.453Z
@Sylvain: I usually prefer to write a short summary of the paper first (as the recommendation often needs to refer to the main contribution), but then I will use sections like "1. Summary of the paper" (very short), "2. Recommendation" (very short), "3. [something else]" (as long as needed), etc. Hence the recommendation will be easy to find.
Jukka Suomela
CC BY-SA 2.5
2011-02-22T18:26:28.417Z
Now i will be able to detect your reviews ;)
Sylvain Peyronnet
CC BY-SA 2.5
2011-02-22T10:55:14.377Z
2011-02-22T10:55:14.377Z
Jukka Suomela
<p>The job of a reviewer is to evaluate whether the paper is worthy of publication. A nice reviewer also helps the authors make their research better. <em>Fixing the paper is not the job of the reviewer.</em> There should be no difference between a review by a junior or senior researchers, though in practice, a senior researcher might spend much less time and effort on it, and will feel much more secure in judging a paper as unworthy.</p>

<p>I agree with <a href="https://cstheory.stackexchange.com/questions/5069/how-to-write-a-negative-review-for-a-conference-paper/5071#5071">@Jukka Suomela</a> that the review needs to be well structured. You want it to be useful to the editor, as well as to the authors, and you want to make it such that the authors cannot weasel their way out of doing what is necessary by "forgetting" to address an issue that is somewhere buried in your long-winded explanation.</p>

<p>This is how I write reviews:</p>

<blockquote>
  <p>The manuscript [title here] presents
  [2-3 sentence summary of the paper to
  demonstrate that you get what the
  paper is about. This demonstrates that
  you actually took the time to read and
  understand.] This is [say something
  good about the paper, even if you
  reject it. The authors will feel
  better]. [Say whether there are major
  flaws, if necessary]. Thus, I
  recommend [state your recommendation -
  unless you're supposed to communicate it separately to the editor, as you may see on some electronic review submission sites (this gives the editor more freedom to override one of the reviewers).]</p>
  
  <p>Major issues</p>
  
  <ol>
  <li><p>[start with a one-sentence "summary" of the issue before you go
  into detail]</p></li>
  <li><p>[By the way, remember that you don't want to rewrite their paper. That's their job. Also, you do not recommend any future work they should do on this project. The issues they need to address are those that allow them to actually make the points that they claim they do. Check your summary if you're not sure]</p></li>
  </ol>
  
  <p>Minor Issues</p>
  
  <p>[keep this short, unless you specifically want to send the message to the editor that the paper is full of little flaws and thus bad. Also, leave out the trivial stuff like typos - instead write e.g. "the paper would benefit from thorough proofreading" - it costs you a lot of time and won't really make a difference in getting the paper published. If you do have a bit of additional time, re-read the major issues again to make sure you're explaining clearly what is wrong and how you expect it to improve. This is a lot more important to both the editor and the authors]</p>
</blockquote>

<p>Note that I would <em>not</em> write out an analytical proof for them. It is their job to write their paper. State "this result should be be proven analytically, for example by following the XX approach". Once you see the paper published somewhere else, and they still don't have an analytical solution, you can contact the authors, saying "Hey, I just saw your paper, and I had an idea for an analytical solution. Are you interested in a collaboration?" </p>

CC BY-SA 2.5
2011-02-22T21:19:18.633Z
I feel you should state minor issues (if you want to help the authors). Otherwise they might be rejected because of those the next time, looping unecessarily.
Raphael
CC BY-SA 2.5
2011-02-22T23:34:24.293Z
@Radu GRIGore, @Raphael: Thanks for your comments. I tried to explain a bit better.
Jonas
CC BY-SA 2.5
2011-02-22T16:46:44.363Z
2011-02-22T23:41:20.710Z
2017-04-13T12:32:28.080Z
Jonas
<p>I think this is too much of a duplicate of the other question, but my comment got too long.</p>

<ul>
<li><p>Firstly, the review process is blind, so you simply have to do the best that you can. Be objective, be polite, give good reasons – see other <a href="https://cstheory.stackexchange.com/questions/1893/how-do-i-referee-a-paper">question</a>. </p></li>
<li><p>There is probably no such thing has having technical comments/improvements that are too long for the standard review, unless you will end up rewriting the paper. Authors tend to appreciate comments, even if they mean more work. If the paper is borderline and is worth improving, then give as many comments as you have time for.</p></li>
<li><p>If you have additional comments that you want to send to the authors, consult with the chair/editor before contacting the authors. Perhaps your comments can be sent anonymously via the chair/editor. </p></li>
<li><p>Papers get rejected, we learn to live with it. The best you can do is provide good comments to help the authors improve the paper for next time round. Authors may get mad, but they don't get even.</p></li>
<li><p>If the paper is terrible, just save your time and don't bother with all the comments. Write a sufficiently convincing review giving reasons why the paper should be rejected, point out that there are many typos and other mistakes, but don't enumerate them all.</p></li>
<li><p>On the other hand, if you are presenting and proving an alternative axiomatisation of their results to the degree that you are not only rewriting their paper, but lifting their results to a whole new level, then you should consult the chair/editor to see what to do. Maybe you should get credit for your contribution. You certainly should <em>never</em> write up your results independently, not until the paper has appeared in print.</p></li>
</ul>

CC BY-SA 2.5
2011-02-22T09:40:20.023Z
2011-02-22T09:59:45.647Z
2017-04-13T12:32:28.077Z
Dave Clarke
<p>This is related to the general question of "<a href="https://cstheory.stackexchange.com/questions/1893/how-do-i-referee-a-paper">How do I referee a paper?</a>". I am reviewing a paper for a conference, and this paper should be rejected, because it is not significant enough for publication and has flaws in some of its technical details. The paper is not wrong, but the ways in which it is right are not very interesting. How do I write a negative review in such a settings? More importantly, how do I do this as a non-senior researcher?</p>

<p>As an example, they make part of their argument through numerics, but I can prove the same result analytically. The analytic treatment is longer than what I can include in a review. I know the author's emails (the review process is not blind both ways), should I email them? This is a conference, so there is no time for revisions, will they be mad at me for rejecting their paper? Should I email them before or after they receive the reviews?</p>

<p>The paper is also sparsely cited, and does not connect strongly to existing literature. I am familiar with many aspects of the relevant literature, how detailed should I make my recommendations on further reading/references?</p>

<p>To extract the general questions:</p>

<ul>
<li>How does a junior researcher write a negative review?</li>
<li>What should you do if you have specific technical comments/improvements that are too long for a standard review?</li>
</ul>

CC BY-SA 2.5
2011-02-22T09:52:14.457Z
It's tempting to answer your question by reproducing the answer given to "How do I referee a paper?".
Jagadish
CC BY-SA 2.5
2011-02-22T12:03:40.600Z
It is well known (whether true or not) that junior researchers review papers more harshly than senior researchers.  Are you reviewing any other papers at the same conference, so you have a baseline for comparison?
Aaron Sterling
CC BY-SA 2.5
2011-02-22T14:26:02.423Z
Be respectful, but brutally honest.
Jeffε
CC BY-SA 2.5
2011-02-22T21:20:16.773Z
Why would a review be depending on your age/reputation? As long as you stick to facts it should not matter at all.
Raphael
CC BY-SA 2.5
2011-04-06T21:12:51.060Z
Flaws in technical details are one thing, but "interesting" is rather subjective.
Rob
<soft-question><paper-review>
How to write a negative review for a conference paper?
Artem Kaznatcheev
# How to write a negative review for a conference paper?

(Asked by: username_0 on 2011-02-22)

This is related to the general question of "[How do I referee a paper?](https://cstheory.stackexchange.com/questions/1893/how-do-i-referee-a-paper)". I am reviewing a paper for a conference, and this paper should be rejected, because it is not significant enough for publication and has flaws in some of its technical details. The paper is not wrong, but the ways in which it is right are not very interesting. How do I write a negative review in such a settings? More importantly, how do I do this as a non-senior researcher?

As an example, they make part of their argument through numerics, but I can prove the same result analytically. The analytic treatment is longer than what I can include in a review. I know the author's emails (the review process is not blind both ways), should I email them? This is a conference, so there is no time for revisions, will they be mad at me for rejecting their paper? Should I email them before or after they receive the reviews?

The paper is also sparsely cited, and does not connect strongly to existing literature. I am familiar with many aspects of the relevant literature, how detailed should I make my recommendations on further reading/references?

To extract the general questions:

* How does a junior researcher write a negative review?
* What should you do if you have specific technical comments/improvements that are too long for a standard review?

**Tags:** soft-question, paper-review
> **Comments:**
> **username_1**: It's tempting to answer your question by reproducing the answer given to "How do I referee a paper?".  
> **username_2**: It is well known (whether true or not) that junior researchers review papers more harshly than senior researchers.  Are you reviewing any other papers at the same conference, so you have a baseline for comparison?  
> **username_3**: Be respectful, but brutally honest.  
> **username_4**: Why would a review be depending on your age/reputation? As long as you stick to facts it should not matter at all.  
> **username_5**: Flaws in technical details are one thing, but "interesting" is rather subjective.  

---

## 3 answers

### Answer by username_9 on 2011-02-22. Score: 20 | Accepted answer

I think this is too much of a duplicate of the other question, but my comment got too long.

* Firstly, the review process is blind, so you simply have to do the best that you can. Be objective, be polite, give good reasons – see other [question](https://cstheory.stackexchange.com/questions/1893/how-do-i-referee-a-paper).
* There is probably no such thing has having technical comments/improvements that are too long for the standard review, unless you will end up rewriting the paper. Authors tend to appreciate comments, even if they mean more work. If the paper is borderline and is worth improving, then give as many comments as you have time for.
* If you have additional comments that you want to send to the authors, consult with the chair/editor before contacting the authors. Perhaps your comments can be sent anonymously via the chair/editor.
* Papers get rejected, we learn to live with it. The best you can do is provide good comments to help the authors improve the paper for next time round. Authors may get mad, but they don't get even.
* If the paper is terrible, just save your time and don't bother with all the comments. Write a sufficiently convincing review giving reasons why the paper should be rejected, point out that there are many typos and other mistakes, but don't enumerate them all.
* On the other hand, if you are presenting and proving an alternative axiomatisation of their results to the degree that you are not only rewriting their paper, but lifting their results to a whole new level, then you should consult the chair/editor to see what to do. Maybe you should get credit for your contribution. You certainly should *never* write up your results independently, not until the paper has appeared in print.


---

### Answer by username_6 on 2011-02-22. Score: 14

> How does a junior researcher write a negative review?

By writing a long, thorough, honest, but polite review.

It helps if you address the *paper*, not the *authors*. Make it as non-personal as possible. Say how the *paper* should be revised, not what the *authors* should do. Say that Theorem 3 is wrong, not that the authors are wrong.

> What should you do if you have specific technical comments/improvements that are too long for a standard review?

There is no such thing as "too long for a review".

Sure, senior researchers write one-line reviews, but that does not mean that a four-page review is too long for a conference paper. If you write a long review, of course anyone will immediately guess that this is written by a junior researcher, but I do not see any reason to avoid it.

---

However, if you write a long review, it is extremely important that it is well-structured and that anyone can find your *recommendation* (i.e., to publish or not, and very briefly why) in two seconds. Make your recommendation stand out, and make sure this part (usually very near the beginning of your review) is very brief.

> **Comments:**
> **username_7**: Start your review by your recommendation : very good advice !  
> **username_6**: @Sylvain: I usually prefer to write a short summary of the paper first (as the recommendation often needs to refer to the main contribution), but then I will use sections like "1. Summary of the paper" (very short), "2. Recommendation" (very short), "3. [something else]" (as long as needed), etc. Hence the recommendation will be easy to find.  
> **username_7**: Now i will be able to detect your reviews ;)  

---

### Answer by username_8 on 2011-02-22. Score: 8

The job of a reviewer is to evaluate whether the paper is worthy of publication. A nice reviewer also helps the authors make their research better. *Fixing the paper is not the job of the reviewer.* There should be no difference between a review by a junior or senior researchers, though in practice, a senior researcher might spend much less time and effort on it, and will feel much more secure in judging a paper as unworthy.

I agree with [@Jukka Suomela](https://cstheory.stackexchange.com/questions/5069/how-to-write-a-negative-review-for-a-conference-paper/5071#5071) that the review needs to be well structured. You want it to be useful to the editor, as well as to the authors, and you want to make it such that the authors cannot weasel their way out of doing what is necessary by "forgetting" to address an issue that is somewhere buried in your long-winded explanation.

This is how I write reviews:

> The manuscript [title here] presents
> [2-3 sentence summary of the paper to
> demonstrate that you get what the
> paper is about. This demonstrates that
> you actually took the time to read and
> understand.] This is [say something
> good about the paper, even if you
> reject it. The authors will feel
> better]. [Say whether there are major
> flaws, if necessary]. Thus, I
> recommend [state your recommendation -
> unless you're supposed to communicate it separately to the editor, as you may see on some electronic review submission sites (this gives the editor more freedom to override one of the reviewers).]
>
> Major issues
>
> 1. [start with a one-sentence "summary" of the issue before you go
>    into detail]
> 2. [By the way, remember that you don't want to rewrite their paper. That's their job. Also, you do not recommend any future work they should do on this project. The issues they need to address are those that allow them to actually make the points that they claim they do. Check your summary if you're not sure]
>
> Minor Issues
>
> [keep this short, unless you specifically want to send the message to the editor that the paper is full of little flaws and thus bad. Also, leave out the trivial stuff like typos - instead write e.g. "the paper would benefit from thorough proofreading" - it costs you a lot of time and won't really make a difference in getting the paper published. If you do have a bit of additional time, re-read the major issues again to make sure you're explaining clearly what is wrong and how you expect it to improve. This is a lot more important to both the editor and the authors]

Note that I would *not* write out an analytical proof for them. It is their job to write their paper. State "this result should be be proven analytically, for example by following the XX approach". Once you see the paper published somewhere else, and they still don't have an analytical solution, you can contact the authors, saying "Hey, I just saw your paper, and I had an idea for an analytical solution. Are you interested in a collaboration?"

> **Comments:**
> **username_4**: I feel you should state minor issues (if you want to help the authors). Otherwise they might be rejected because of those the next time, looping unecessarily.  
> **username_8**: @Radu GRIGore, @Raphael: Thanks for your comments. I tried to explain a bit better.  

---

