<p>The main concern has always been the "binning" of quality scores that occurs via CRAM compression (and is also standard on the HiSeqX, HiSeq4000, and NovaSeq platforms). Anecdotally, I can report very little difference between 4-bin quality scores and full quality scores on cancer samples, though I don't know if I've seen a direct head-to-head comparison. </p>

CC BY-SA 3.0
2017-06-09T16:32:37.037Z
+1. My colleagues have done some benchmarks to show 4-bin has little effect on *germline* samples. I have seen the similar. Cancer samples always make me wary, though. It would be great if someone do a systematic evaluation on cancer samples. I haven't seen one so far.
user172818
CC BY-SA 3.0
2017-06-09T16:28:26.360Z
2017-06-09T16:28:26.360Z
chrisamiller
<p>By default, a CRAM you create with samtools is lossless. It typically halves the input BAM in terms of file size. If you want to compress more, you can let samtools convert most read names to integers. You won't be able to tell optical duplicates from read names, but this is a minor concern. You can also drop useless tags depending on your mapper and the downstream caller in use. For cancer data, I wouldn't reduce the resolution of base quality without comprehensive benchmarks. Unfortunately, base quality takes most of space in CRAM. Discarding the original read names and some tags probably won't save you much space.</p>

CC BY-SA 3.0
2017-06-08T16:24:48.840Z
These are all great suggestions for reducing file size without losing information, but don't address the main question: the effect of lossiness on variant calls.
Daniel Standage
CC BY-SA 3.0
2017-06-08T16:27:14.510Z
@DanielS If you don't touch bases and qualities and name pairing, you won't change the variant calls.
user172818
CC BY-SA 3.0
2017-06-08T16:30:41.023Z
Yes, but then that's not really lossy, is it? Doesn't lossy compression conventionally involve changing the sequence and/or quality values for greater compression efficiency?
Daniel Standage
CC BY-SA 3.0
2017-06-08T16:41:27.150Z
That depends on the definition of "lossy" :) To me, losing read names and tags is lossy.
user172818
CC BY-SA 3.0
2017-06-08T16:49:53.817Z
¯\\_(ツ)_/¯ You yourself said that calls shouldn't change if the sequence and quality are unchanged. So everything else is ancillary. Don't get me wrong, I think it's valuable to point out that it's possible to reduce file size without changing sequence or quality, but it seemed pretty clear to me that the OP was talking about lossy compression of the sequence and/or quality scores.
Daniel Standage
CC BY-SA 3.0
2017-06-08T16:51:38.977Z
Then again, this answer is a reasonable response to the question "Which compression strategy has the lowest impact?" Ok, I take it all back! :-)
Daniel Standage
CC BY-SA 3.0
2017-06-08T16:20:00.480Z
2017-06-08T16:20:00.480Z
user172818
<p>We're considering switching our storage format from BAM to CRAM. We work with human cancer samples, which may have very low prevalence variants (i.e. not diploid frequency).</p>

<p>If we use lossy CRAM to save more space, how much will variants called from those CRAM files change? Which compression strategy has the lowest impact? </p>

<p>Are there any other impacts on downstream tools that we're not considering?</p>

CC BY-SA 3.0
2017-06-08T14:58:54.587Z
CRAM doesn't **need** to be lossy, is there a reason you need it to be?
Devon Ryan
CC BY-SA 3.0
2017-06-08T15:02:49.157Z
Saving disk space. We pay by the GB and need to keep the data around for 10 years.
morgantaschuk
CC BY-SA 3.0
2017-06-08T15:08:59.283Z
Can't argue that budget isn't a good reason :)
Devon Ryan
CC BY-SA 3.0
2017-06-08T15:16:31.463Z
Interesting question. I think this is the kind of thing that makes a nice side project. Take a bam file call the variants, transform it to cram and run the variant caller. Measure the difference and the variant concordance between the two approaches using a number of different files.
eastafri
CC BY-SA 3.0
2017-06-08T16:54:31.743Z
If one is concerned about absolute integrity/reproducibility of the data then budget consideration is not a good reason.
GenoMax
CC BY-SA 3.0
2017-06-08T22:08:54.607Z
Unless the answer is: some types of lossy compression don't have any impact on variant calls.
morgantaschuk
CC BY-SA 3.0
2017-06-08T22:32:15.993Z
BAM files are zipped with standard gzip compression. Unzip them to "naked BAM" - not my terminology - and re-zip them with something stronger like 7zip/LZMA. You can always re-zip them again with the bgzip tool when you need them back in true BAM format again. This gets you most of the way there filesize-wise without really changing the format, which could be good if you've got things set up how you like. Not an answer because it doesn't answer your question but it might solve your problem.
J.J
<sam><file-formats><variant-calling><storage>
Do variant calls change when you call from CRAM?
# Do variant calls change when you call from CRAM?

(Asked by: username_0 on 2017-06-08)

We're considering switching our storage format from BAM to CRAM. We work with human cancer samples, which may have very low prevalence variants (i.e. not diploid frequency).

If we use lossy CRAM to save more space, how much will variants called from those CRAM files change? Which compression strategy has the lowest impact?

Are there any other impacts on downstream tools that we're not considering?

**Tags:** sam, file-formats, variant-calling, storage
> **Comments:**
> **username_1**: CRAM doesn't **need** to be lossy, is there a reason you need it to be?  
> **username_0**: Saving disk space. We pay by the GB and need to keep the data around for 10 years.  
> **username_1**: Can't argue that budget isn't a good reason :)  
> **username_2**: Interesting question. I think this is the kind of thing that makes a nice side project. Take a bam file call the variants, transform it to cram and run the variant caller. Measure the difference and the variant concordance between the two approaches using a number of different files.  
> **username_3**: If one is concerned about absolute integrity/reproducibility of the data then budget consideration is not a good reason.  
> **username_0**: Unless the answer is: some types of lossy compression don't have any impact on variant calls.  
> **username_4**: BAM files are zipped with standard gzip compression. Unzip them to "naked BAM" - not my terminology - and re-zip them with something stronger like 7zip/LZMA. You can always re-zip them again with the bgzip tool when you need them back in true BAM format again. This gets you most of the way there filesize-wise without really changing the format, which could be good if you've got things set up how you like. Not an answer because it doesn't answer your question but it might solve your problem.  

---

## 2 answers

### Answer by username_6 on 2017-06-08. Score: 6 | Accepted answer

By default, a CRAM you create with samtools is lossless. It typically halves the input BAM in terms of file size. If you want to compress more, you can let samtools convert most read names to integers. You won't be able to tell optical duplicates from read names, but this is a minor concern. You can also drop useless tags depending on your mapper and the downstream caller in use. For cancer data, I wouldn't reduce the resolution of base quality without comprehensive benchmarks. Unfortunately, base quality takes most of space in CRAM. Discarding the original read names and some tags probably won't save you much space.

> **Comments:**
> **username_7**: These are all great suggestions for reducing file size without losing information, but don't address the main question: the effect of lossiness on variant calls.  
> **username_6**: @DanielS If you don't touch bases and qualities and name pairing, you won't change the variant calls.  
> **username_7**: Yes, but then that's not really lossy, is it? Doesn't lossy compression conventionally involve changing the sequence and/or quality values for greater compression efficiency?  
> **username_6**: That depends on the definition of "lossy" :) To me, losing read names and tags is lossy.  
> **username_7**: ¯\\_(ツ)_/¯ You yourself said that calls shouldn't change if the sequence and quality are unchanged. So everything else is ancillary. Don't get me wrong, I think it's valuable to point out that it's possible to reduce file size without changing sequence or quality, but it seemed pretty clear to me that the OP was talking about lossy compression of the sequence and/or quality scores.  
> **username_7**: Then again, this answer is a reasonable response to the question "Which compression strategy has the lowest impact?" Ok, I take it all back! :-)  

---

### Answer by username_5 on 2017-06-09. Score: 3

The main concern has always been the "binning" of quality scores that occurs via CRAM compression (and is also standard on the HiSeqX, HiSeq4000, and NovaSeq platforms). Anecdotally, I can report very little difference between 4-bin quality scores and full quality scores on cancer samples, though I don't know if I've seen a direct head-to-head comparison.

> **Comments:**
> **username_6**: +1. My colleagues have done some benchmarks to show 4-bin has little effect on *germline* samples. I have seen the similar. Cancer samples always make me wary, though. It would be great if someone do a systematic evaluation on cancer samples. I haven't seen one so far.  

---

