<p>The problem is that the enrichment score is modified based on whether the sets are positively or negatively associated, which means that an unbalanced set via either associative statistic or gene count (as is almost always the case) will lead to an incorrect total score.</p>
<p>Put in other words, the calculated score from the positive genes alters the calculated score from the negative genes.</p>
<p>Given this, it's a better idea to either separately calculate scores for the positive set and the negative set (this is the easiest approach, and works with existing methods), or (equivalently) recalculate enrichment score modifiers for positive and negative genes separately (i.e. adjust the graph) so that the zero point for the enrichment score crosses the zero point for association.</p>
<p>A common argument against this (when I have mentioned it to others) is that GSEA has <em>always</em> been done on the total gene set, so it makes sense to do it on the total gene set because it has always been done that way. I'm not a huge fan of the status-quo argument.</p>
<p>I alluded to this issue in <a href="https://bioinformatics.stackexchange.com/a/15963/73">this previous answer</a>.</p>
<p>With regards to your figures:</p>
<p><strong>a</strong>. Zero-point crossover is at an enrichment score of ~0.42, so the negative set will be underemphasised. It's not contributing much anyway (except for the one most negative gene at the end), so wouldn't affect the outcome much.</p>
<p><strong>b</strong>. Zero-point crossover is at an enrichment score of ~0.45. Similar situation as in <strong>a</strong>.</p>
<p><strong>c</strong>. Zero-point crossover is at an enrichment score of ~0.32. There aren't really any positive genes with substantial association scores, so no issue.</p>
<p><strong>d</strong>. Plot looks to be identical to <strong>c</strong>, so the same comments apply.</p>
<p>The scores are most affected when they have a lot of area on both sides of the Y axis. The top right plot of this sub-figure is an example of that, showing a sharp-ish tail on the negative side, so the calculated enrichment score for the negative peak is less extreme than it would be if only the negative genes were considered [<a href="https://www.nature.com/articles/s41590-021-01067-0/figures/8" rel="nofollow noreferrer">source</a>]:</p>
<p><a href="https://i.sstatic.net/gYveS.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/gYveS.jpg" alt="GSEA plots with zero points marked; most plots show a large steep curve for positive genes, with a small hook for negative genes" /></a></p>
<blockquote>
<p>The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.</p>
</blockquote>
<p>Yes. If the positive genes are weighted separately from the negative genes, then the bias can be removed. I did briefly attempt a non-parametric weighting for GSEA (based on rank), but got too confused by the maths involved. Someone answered my question about it, but I haven't had to do any GSEA since then, so haven't actually verified if their solution works for me:</p>
<p><a href="https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h">https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h</a></p>

CC BY-SA 4.0
2023-03-22T01:37:42.903Z
Ah I believe I see your point now about recalibrating the graph I think. In practice, I'm guessing this would be done by having N - N_H in P_miss be equal to the "missing genes" on the positive side ONLY, for positive genes, and negative side ONLY, for negative genes
An Ignorant Wanderer
CC BY-SA 4.0
2023-03-22T01:40:05.220Z
Analogously we'd need to modify P_hit so the denominator only has genes on positive side vs negative side
An Ignorant Wanderer
CC BY-SA 4.0
2023-03-21T21:00:04.993Z
2023-05-09T21:17:54.697Z
2023-05-09T21:17:54.697Z
gringer
<p>I'm reading Subramanian et. al's (2005) original GSEA <a href="https://www.pnas.org/doi/10.1073/pnas.0506580102" rel="nofollow noreferrer">paper</a>. In the paper, the authors mention the following:</p>
<blockquote>
<p>We noticed that the use of
weighted steps could cause the distribution of observed ES scores to
be asymmetric in cases where many more genes are correlated with one
of the two phenotypes. We therefore estimate the significance levels
by considering separately the positively and negatively scoring gene
sets</p>
</blockquote>
<p>I understand what they're saying, but I simply don't understand why it's a problem. Basically, they're saying that in some cases, most genes in the list we're testing could appear mostly at the top or bottom of the list. But I don't understand why having asymmetric distributions is a problem. What is the issue here?</p>
<p>EDIT: Here are some example GSEA plots I found via a web search with such gene sets. Where is the problem?
<a href="https://i.sstatic.net/qJdzT.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qJdzT.jpg" alt="enter image description here" /></a></p>
<p>One thing I don't get though is why this problem would be introduced through &quot;weighting&quot; of the steps. I feel like it would also be a problem without weighting. The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.</p>

<gsea>
Why are asymmetric distributions an issue for GSEA?
# Why are asymmetric distributions an issue for GSEA?

(Asked by: username_0 on 2023-03-21)

I'm reading Subramanian et. al's (2005) original GSEA [paper](https://www.pnas.org/doi/10.1073/pnas.0506580102). In the paper, the authors mention the following:

> We noticed that the use of
> weighted steps could cause the distribution of observed ES scores to
> be asymmetric in cases where many more genes are correlated with one
> of the two phenotypes. We therefore estimate the significance levels
> by considering separately the positively and negatively scoring gene
> sets

I understand what they're saying, but I simply don't understand why it's a problem. Basically, they're saying that in some cases, most genes in the list we're testing could appear mostly at the top or bottom of the list. But I don't understand why having asymmetric distributions is a problem. What is the issue here?

EDIT: Here are some example GSEA plots I found via a web search with such gene sets. Where is the problem?
[![enter image description here](https://i.sstatic.net/qJdzT.jpg)](https://i.sstatic.net/qJdzT.jpg)

One thing I don't get though is why this problem would be introduced through "weighting" of the steps. I feel like it would also be a problem without weighting. The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.

**Tags:** gsea

---

## 1 answers

### Answer by username_1 on 2023-03-21. Score: 1 | Accepted answer

The problem is that the enrichment score is modified based on whether the sets are positively or negatively associated, which means that an unbalanced set via either associative statistic or gene count (as is almost always the case) will lead to an incorrect total score.

Put in other words, the calculated score from the positive genes alters the calculated score from the negative genes.

Given this, it's a better idea to either separately calculate scores for the positive set and the negative set (this is the easiest approach, and works with existing methods), or (equivalently) recalculate enrichment score modifiers for positive and negative genes separately (i.e. adjust the graph) so that the zero point for the enrichment score crosses the zero point for association.

A common argument against this (when I have mentioned it to others) is that GSEA has *always* been done on the total gene set, so it makes sense to do it on the total gene set because it has always been done that way. I'm not a huge fan of the status-quo argument.

I alluded to this issue in [this previous answer](https://bioinformatics.stackexchange.com/a/15963/73).

With regards to your figures:

**a**. Zero-point crossover is at an enrichment score of ~0.42, so the negative set will be underemphasised. It's not contributing much anyway (except for the one most negative gene at the end), so wouldn't affect the outcome much.

**b**. Zero-point crossover is at an enrichment score of ~0.45. Similar situation as in **a**.

**c**. Zero-point crossover is at an enrichment score of ~0.32. There aren't really any positive genes with substantial association scores, so no issue.

**d**. Plot looks to be identical to **c**, so the same comments apply.

The scores are most affected when they have a lot of area on both sides of the Y axis. The top right plot of this sub-figure is an example of that, showing a sharp-ish tail on the negative side, so the calculated enrichment score for the negative peak is less extreme than it would be if only the negative genes were considered [[source](https://www.nature.com/articles/s41590-021-01067-0/figures/8)]:

[![GSEA plots with zero points marked; most plots show a large steep curve for positive genes, with a small hook for negative genes](https://i.sstatic.net/gYveS.jpg)](https://i.sstatic.net/gYveS.jpg)

> The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.

Yes. If the positive genes are weighted separately from the negative genes, then the bias can be removed. I did briefly attempt a non-parametric weighting for GSEA (based on rank), but got too confused by the maths involved. Someone answered my question about it, but I haven't had to do any GSEA since then, so haven't actually verified if their solution works for me:

<https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h>

> **Comments:**
> **username_0**: Ah I believe I see your point now about recalibrating the graph I think. In practice, I'm guessing this would be done by having N - N_H in P_miss be equal to the "missing genes" on the positive side ONLY, for positive genes, and negative side ONLY, for negative genes  
> **username_0**: Analogously we'd need to modify P_hit so the denominator only has genes on positive side vs negative side  

---

