<p>If you want a statistical reason not to use pseudobulk to do DE for marker finding, I think it is the same reason not to use any DE method for marker finding: the <strong>double dipping problem</strong> (see <a href="https://arxiv.org/abs/2012.02936" rel="nofollow noreferrer">Gao and Witten 2020</a>).</p>
<p>Briefly, if you cluster a dataset based on genes (we can safely say that clustering in PCA space is <em>still</em> based on genes), then asking which genes are differentially expressed between clusters is a statistical fallacy.
We could just as easily divide apples in 2 groups according to their shade of red and then ask which RGB channels exhibit a statistically significant difference.</p>
<p>This means that even in the absence of real structure, you will identify very low p-values for your DE test. The OSCA book section you mention links to another <a href="http://bioconductor.org/books/3.14/OSCA.advanced/marker-detection-redux.html#p-value-invalidity" rel="nofollow noreferrer">section</a> in the advanced book that tells you the same thing with a small practical demonstration generating a 200-dimensional Gaussian and clustering it, then testing for differences in clusters.</p>
<p>This does <em>not</em> mean, however, that markers found this way are meaningless; it simply means that you cannot use a p-value cutoff to define whether a gene is a marker in this scenario, or say that since you have statistically significant markers two clusters are different.</p>
<p>You can still rank these markers by fold change, difference in percentage expression, AUC, entropy, etc <em>as well as p-value</em>. Moreover, markers identified using the traditional methods tend to make biological sense, and statistically rigorous methods are still in development (there was a beautiful talk by Anna Neufeld on an approach involving count splitting, you can read the preprint <a href="https://arxiv.org/abs/2207.00554" rel="nofollow noreferrer">here</a>).</p>
<p>The test you suggest is interesting, but I don't think you would solve the double dipping problem; this being said it looks more robust and appropriate than using every single cell as a replicate.</p>

CC BY-SA 4.0
2022-12-09T19:21:33.687Z
Thank you for the useful addition, indeed the pseudobulk approach does not solve the double-dipping problem, and will suffer from the same p-value inflation as a Wilcoxon on the single cells.
Alexlok
CC BY-SA 4.0
2022-12-08T22:32:15.720Z
2022-12-08T22:32:15.720Z
gdagstn
<p>A standard approach for scRNA-Seq is to partition the single cells into individual clusters, then use a Wilcoxon test to find markers that characterize each cluster (or other statistical methods that consider single cells as replicates).</p>
<p>Recently, there has been a push towards using pseudobulk approaches for &quot;differential state&quot; (DS) analysis, which avoid the pseudoreplication bias of considering single cells as replicates. See e.g. <a href="https://www.nature.com/articles/s41467-020-19894-4" rel="nofollow noreferrer">Crowell (2020)</a>, <a href="https://www.nature.com/articles/s41467-021-21038-1" rel="nofollow noreferrer">Zimmerman (2021)</a>, <a href="https://academic.oup.com/bioinformatics/article/37/19/3243/6273181" rel="nofollow noreferrer">Thurman (2021)</a>, <a href="https://www.nature.com/articles/s41467-021-25960-2" rel="nofollow noreferrer">Squair (2021)</a>, or the <a href="http://bioconductor.org/books/3.14/OSCA.multisample/multi-sample-comparisons.html#testing-for-between-label-differences" rel="nofollow noreferrer">OSCA book</a>.</p>
<p>However, all the above papers seem to use pseudobulk methods for DS of a single cluster between two conditions (e.g. drug treatment), or occasionally to compare a specific pair of clusters. I haven't seen any example of using pseudobulk to find markers distinguishing clusters (comparisons of one cluster against all others).</p>
<p>I can think of a few reasons, for example:</p>
<ul>
<li>these pseudobulk methods can be more computationally intensive,</li>
<li>perhaps the Wilcoxon test is good enough to simply find markers,</li>
<li>pseudobulk methods can only work when each cluster is represented in several samples, which is hard to guarantee for every single cluster in the dataset.</li>
</ul>
<p>On the other hand it seems to me a pseudobulk approach should yield more relevant markers (genes that are robustly enriched across biological replicates). So, am I missing something? Is there a statistical reason not to use pseudobulk approaches for finding cluster-specific markers?</p>

<scrnaseq><differential-expression>
scRNA-Seq pseudobulk for marker detection
# scRNA-Seq pseudobulk for marker detection

(Asked by: username_0 on 2022-04-26)

A standard approach for scRNA-Seq is to partition the single cells into individual clusters, then use a Wilcoxon test to find markers that characterize each cluster (or other statistical methods that consider single cells as replicates).

Recently, there has been a push towards using pseudobulk approaches for "differential state" (DS) analysis, which avoid the pseudoreplication bias of considering single cells as replicates. See e.g. [Crowell (2020)](https://www.nature.com/articles/s41467-020-19894-4), [Zimmerman (2021)](https://www.nature.com/articles/s41467-021-21038-1), [Thurman (2021)](https://academic.oup.com/bioinformatics/article/37/19/3243/6273181), [Squair (2021)](https://www.nature.com/articles/s41467-021-25960-2), or the [OSCA book](http://bioconductor.org/books/3.14/OSCA.multisample/multi-sample-comparisons.html#testing-for-between-label-differences).

However, all the above papers seem to use pseudobulk methods for DS of a single cluster between two conditions (e.g. drug treatment), or occasionally to compare a specific pair of clusters. I haven't seen any example of using pseudobulk to find markers distinguishing clusters (comparisons of one cluster against all others).

I can think of a few reasons, for example:

* these pseudobulk methods can be more computationally intensive,
* perhaps the Wilcoxon test is good enough to simply find markers,
* pseudobulk methods can only work when each cluster is represented in several samples, which is hard to guarantee for every single cluster in the dataset.

On the other hand it seems to me a pseudobulk approach should yield more relevant markers (genes that are robustly enriched across biological replicates). So, am I missing something? Is there a statistical reason not to use pseudobulk approaches for finding cluster-specific markers?

**Tags:** scrnaseq, differential-expression

---

## 1 answers

### Answer by username_1 on 2022-12-08. Score: 1

If you want a statistical reason not to use pseudobulk to do DE for marker finding, I think it is the same reason not to use any DE method for marker finding: the **double dipping problem** (see [Gao and Witten 2020](https://arxiv.org/abs/2012.02936)).

Briefly, if you cluster a dataset based on genes (we can safely say that clustering in PCA space is *still* based on genes), then asking which genes are differentially expressed between clusters is a statistical fallacy.
We could just as easily divide apples in 2 groups according to their shade of red and then ask which RGB channels exhibit a statistically significant difference.

This means that even in the absence of real structure, you will identify very low p-values for your DE test. The OSCA book section you mention links to another [section](http://bioconductor.org/books/3.14/OSCA.advanced/marker-detection-redux.html#p-value-invalidity) in the advanced book that tells you the same thing with a small practical demonstration generating a 200-dimensional Gaussian and clustering it, then testing for differences in clusters.

This does *not* mean, however, that markers found this way are meaningless; it simply means that you cannot use a p-value cutoff to define whether a gene is a marker in this scenario, or say that since you have statistically significant markers two clusters are different.

You can still rank these markers by fold change, difference in percentage expression, AUC, entropy, etc *as well as p-value*. Moreover, markers identified using the traditional methods tend to make biological sense, and statistically rigorous methods are still in development (there was a beautiful talk by Anna Neufeld on an approach involving count splitting, you can read the preprint [here](https://arxiv.org/abs/2207.00554)).

The test you suggest is interesting, but I don't think you would solve the double dipping problem; this being said it looks more robust and appropriate than using every single cell as a replicate.

> **Comments:**
> **username_0**: Thank you for the useful addition, indeed the pseudobulk approach does not solve the double-dipping problem, and will suffer from the same p-value inflation as a Wilcoxon on the single cells.  

---

