<p>I think since your PCA shows that your normal samples cluster right alongside the tumor ones, you have to conclude that there is much more variation between tumors than between tumors and controls.  DESeq's differentially expressed gene list was generated properly.</p>

<p>Your results are what they are, sorry they are so useless.</p>

CC BY-SA 4.0
2018-07-10T17:15:37.113Z
2018-07-10T17:15:37.113Z
swbarnes2
<p>In general, messing with FDR or log fold change cutoffs beyond the ones you used gets pretty sketchy. I usually only adjust these values in the more conservative direction, although you may be able to get away with FDR &lt; 0.1 if you have a large effect size (which you don't).</p>

<p>You might consider investigating why you don't seem to get any differentially expressed genes before adjusting your cutoffs: PCA on the samples based on their top 1,000 most variably expressed genes would be a good start. Do the tumor and normal transcriptomes cluster apart from each other?</p>

<p>With so few normal samples, it is possible that you are not fully characterizing the variability of that population and it is affecting your ability to identify meaningful differences between normal and tumor.</p>

CC BY-SA 4.0
2018-07-09T22:57:14.243Z
Hi, I made the PCA and it is given in the question. please have a look.
beginner
CC BY-SA 4.0
2018-07-19T13:25:27.553Z
If you have a large effect size you wouldn’t need to increase the FDR to 0.1: a lower FDR would still capture the effect.
Konrad Rudolph
CC BY-SA 4.0
2018-07-09T22:45:55.040Z
2018-07-09T22:45:55.040Z
Ben D.
<ol>
<li><p>You can usually get away with FDR &lt; 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called "false discovery rate" for a reason.</p></li>
<li><p>Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.</p></li>
<li><p>You're just going to waste your time if you ignore FDR.</p></li>
</ol>

<p>Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.</p>

<p>It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.</p>

CC BY-SA 4.0
2018-07-10T13:13:06.150Z
Thanks Devon. Yes there will be follow up experiments after selection. So, what would recommend me now? Can I go with FC and FDR < 0.1? Yes, there is batch effect what can I do now? Can I apply subsampling procedure to get more DEGs?
beginner
CC BY-SA 4.0
2018-07-10T13:14:40.237Z
Yes, FDR<0.1 and ideally some reasonable fold-change (say a 30% change). Have a look at packages like `sva` for dealing with your batch effect.
Devon Ryan
CC BY-SA 4.0
2018-07-10T13:35:04.717Z
Is it a good idea to apply subsampling procedure for DEA?
beginner
CC BY-SA 4.0
2018-07-10T14:21:22.020Z
@beginner No, that's not useful, presuming you mean subsampling from the counts.
Devon Ryan
CC BY-SA 4.0
2018-07-10T14:28:32.870Z
selecting random samples from cancer cohort and repeating differential analysis n times will give different set of DEGs and then select which DEGs are most repeated in all the analyses. This is the one I'm asking.
beginner
CC BY-SA 4.0
2018-07-10T14:30:14.933Z
No, that's not helpful, you're just approximating the results you would have gotten otherwise.
Devon Ryan
CC BY-SA 4.0
2018-07-10T14:42:09.317Z
Sure thank you. And one last question. After selecting candidates..for experiments I have selected two cell-lines.To look which gene is more expressed in which cell line, doing a simple t-test between the cell-lines for those genes is fine or should I go with Deseq2 to check how those genes were expressed?
beginner
CC BY-SA 4.0
2018-07-10T14:47:11.397Z
Use DESeq2, it's more powerful than a T-test.
Devon Ryan
CC BY-SA 4.0
2018-07-10T15:05:57.167Z
Yes, it is more powerful but Deseq2 needs replicates for performing differential analysis. It will give a warning/error if tried to analyze data without replicates.
beginner
CC BY-SA 4.0
2018-07-10T15:08:38.413Z
I assumed you were talking about cell lines with replicates. No method will reliably work without replicates.
Devon Ryan
CC BY-SA 4.0
2018-07-10T15:15:32.023Z
No, I just have two breast cell-lines. Interested in doing experiments using one cell-line. So I thought of looking at candidates expression in those cell-lines and then select based on that.
beginner
CC BY-SA 4.0
2018-07-10T15:41:53.520Z
@beginner It’s the other way round: DESeq needs *fewer* replicates than a standard t-test, because it pools variance estimates. This is, in fact, the whole reason why dedicated DE analysis packages exist. With 45 samples a simple t-test should actually work just as well as DESeq though.
Konrad Rudolph
CC BY-SA 4.0
2018-07-10T15:44:31.460Z
@KonradRudolph you have mistaken. please check my previous comments.
beginner
CC BY-SA 4.0
2018-07-10T15:50:38.593Z
@beginner I’m puzzled. First off, your question mentions 45 replicates, not two (or one). Secondly, how do you suggest performing a t-test on two replicates?! (One replicate doesn’t even make sense … what would you compare against?)
Konrad Rudolph
CC BY-SA 4.0
2018-07-10T15:55:10.473Z
No. After doing differential analysis between 40 tumor and 5 normal I get some differential expressed genes. From that I will select strong candidates and do some experiments. For the experiments I have two cell-lines. To select the good cell-line where I can see high expression of those candidates I have to do differential analysis between those two cell-lines. Note: Please have a look at my comments and Devon's comments.
beginner
CC BY-SA 4.0
2018-07-10T13:07:58.930Z
2018-07-10T13:07:58.930Z
Devon Ryan
<p>I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with <code>Deseq2</code> based on Fold change > 1.2 and alpha &lt; 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes. </p>

<pre><code>res &lt;- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)
</code></pre>

<p>To get more number of differential expressed genes I have few questions now </p>

<ol>
<li><p>Instead of FDR &lt; 0.05 can I use FDR &lt; 0.1 (or) FDR &lt; 0.5. Will there be any low confidence with this?</p></li>
<li><p>Can I select differential expressed genes only based on FDR &lt; 0.05 without any fold change cutoff?</p></li>
<li><p>As I get very low number of DEGs with FDR &lt; 0.05 &amp; Foldchange > 1.2, Can I select DEGs based on Foldchange > 1.2 and p.value &lt; 0.01 or 0.05 ?</p></li>
</ol>

<p>PCA of the samples looks like this:</p>

<p><a href="https://i.sstatic.net/Gfxbq.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Gfxbq.png" alt="PCA plot Tumor vs Normal"></a></p>

CC BY-SA 4.0
2018-07-10T15:38:27.450Z
(FDR) ɑ < 0.5 would be completely meaningless. With 40 tumour and 5 normal samples you should get extremely robust results, and I’d suggest that you should use something like ɑ < 0.0001. Given your PCA and your lack of DE, the only valid conclusion is that you really have *no evidence whatsoever* for differential expression. The samples are indistinguishable by the state of the art. This in itself is a striking finding.
Konrad Rudolph
<r><rna-seq><differential-expression><deseq2><rna>
Selection of differential expressed genes
# Selection of differential expressed genes

(Asked by: username_0 on 2018-07-09)

I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with Deseq2 based on Fold change > 1.2 and alpha < 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes. 
res <- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)

To get more number of differential expressed genes I have few questions now 

Instead of FDR < 0.05 can I use FDR < 0.1 (or) FDR < 0.5. Will there be any low confidence with this?
Can I select differential expressed genes only based on FDR < 0.05 without any fold change cutoff?
As I get very low number of DEGs with FDR < 0.05 & Foldchange > 1.2, Can I select DEGs based on Foldchange > 1.2 and p.value < 0.01 or 0.05 ?

PCA of the samples looks like this:

**Tags:** r, rna-seq, differential-expression, deseq2, rna
> **Comments:**
> **username_1**: (FDR) ɑ < 0.5 would be completely meaningless. With 40 tumour and 5 normal samples you should get extremely robust results, and I’d suggest that you should use something like ɑ < 0.0001. Given your PCA and your lack of DE, the only valid conclusion is that you really have *no evidence whatsoever* for differential expression. The samples are indistinguishable by the state of the art. This in itself is a striking finding.  

---

## 3 answers

### Answer by username_4 on 2018-07-10. Score: 3 | Accepted answer

1. You can usually get away with FDR < 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called "false discovery rate" for a reason.
2. Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.
3. You're just going to waste your time if you ignore FDR.

Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.

It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.

> **Comments:**
> **username_0**: Thanks Devon. Yes there will be follow up experiments after selection. So, what would recommend me now? Can I go with FC and FDR < 0.1? Yes, there is batch effect what can I do now? Can I apply subsampling procedure to get more DEGs?  
> **username_4**: Yes, FDR<0.1 and ideally some reasonable fold-change (say a 30% change). Have a look at packages like `sva` for dealing with your batch effect.  
> **username_0**: Is it a good idea to apply subsampling procedure for DEA?  
> **username_4**: @beginner No, that's not useful, presuming you mean subsampling from the counts.  
> **username_0**: selecting random samples from cancer cohort and repeating differential analysis n times will give different set of DEGs and then select which DEGs are most repeated in all the analyses. This is the one I'm asking.  
> **username_4**: No, that's not helpful, you're just approximating the results you would have gotten otherwise.  
> **username_0**: Sure thank you. And one last question. After selecting candidates..for experiments I have selected two cell-lines.To look which gene is more expressed in which cell line, doing a simple t-test between the cell-lines for those genes is fine or should I go with Deseq2 to check how those genes were expressed?  
> **username_4**: Use DESeq2, it's more powerful than a T-test.  
> **username_0**: Yes, it is more powerful but Deseq2 needs replicates for performing differential analysis. It will give a warning/error if tried to analyze data without replicates.  
> **username_4**: I assumed you were talking about cell lines with replicates. No method will reliably work without replicates.  
> **username_0**: No, I just have two breast cell-lines. Interested in doing experiments using one cell-line. So I thought of looking at candidates expression in those cell-lines and then select based on that.  
> **username_1**: @beginner It’s the other way round: DESeq needs *fewer* replicates than a standard t-test, because it pools variance estimates. This is, in fact, the whole reason why dedicated DE analysis packages exist. With 45 samples a simple t-test should actually work just as well as DESeq though.  
> **username_0**: @KonradRudolph you have mistaken. please check my previous comments.  
> **username_1**: @beginner I’m puzzled. First off, your question mentions 45 replicates, not two (or one). Secondly, how do you suggest performing a t-test on two replicates?! (One replicate doesn’t even make sense … what would you compare against?)  
> **username_0**: No. After doing differential analysis between 40 tumor and 5 normal I get some differential expressed genes. From that I will select strong candidates and do some experiments. For the experiments I have two cell-lines. To select the good cell-line where I can see high expression of those candidates I have to do differential analysis between those two cell-lines. Note: Please have a look at my comments and Devon's comments.  

---

### Answer by username_2 on 2018-07-10. Score: 1

I think since your PCA shows that your normal samples cluster right alongside the tumor ones, you have to conclude that there is much more variation between tumors than between tumors and controls. DESeq's differentially expressed gene list was generated properly.

Your results are what they are, sorry they are so useless.


---

### Answer by username_3 on 2018-07-09. Score: 0

In general, messing with FDR or log fold change cutoffs beyond the ones you used gets pretty sketchy. I usually only adjust these values in the more conservative direction, although you may be able to get away with FDR < 0.1 if you have a large effect size (which you don't).

You might consider investigating why you don't seem to get any differentially expressed genes before adjusting your cutoffs: PCA on the samples based on their top 1,000 most variably expressed genes would be a good start. Do the tumor and normal transcriptomes cluster apart from each other?

With so few normal samples, it is possible that you are not fully characterizing the variability of that population and it is affecting your ability to identify meaningful differences between normal and tumor.

> **Comments:**
> **username_0**: Hi, I made the PCA and it is given in the question. please have a look.  
> **username_1**: If you have a large effect size you wouldn’t need to increase the FDR to 0.1: a lower FDR would still capture the effect.  

---

