<p><em>Answer from @alex-reynolds, converted from comments:</em></p>
<p>As with any compression scheme, if you have to store a reference genome, along with the 4 Mb &quot;diff&quot;, is it really compressed?</p>
<p><a href="http://mattmahoney.net/dc/dce.html" rel="nofollow noreferrer">http://mattmahoney.net/dc/dce.html</a></p>
<p>For sequencing data, you might also look at the CRAM format, which also uses diffs against a reference genome to store reduced datasets. Labs are switching from BAM to CRAM to manage storage of increased numbers of sequencing datasets, with associated changes to pipelines and toolkits to use this format. For performance reasons, BAM may be preferred for analysis, and CRAM for long-term storage.</p>
<p><a href="https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats" rel="nofollow noreferrer">https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats</a></p>
<p><a href="https://www.ga4gh.org/news/cram-compression-for-genomics/" rel="nofollow noreferrer">https://www.ga4gh.org/news/cram-compression-for-genomics/</a></p>

CC BY-SA 4.0
2021-07-14T11:08:14.203Z
2021-07-14T11:08:14.203Z
gringer
<p>I saw this paper a few weeks back that claims to offer 4x more compression than typical formats for both fastqs and BAM/CRAM files.</p>
<p>Genozip - A Universal Extensible Genomic Data Compressor</p>
<p><a href="https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077" rel="nofollow noreferrer">https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077</a></p>

CC BY-SA 4.0
2021-08-13T13:35:31.273Z
2021-08-13T13:35:31.273Z
story
<p>Multiple sources (see for instance <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5958914/" rel="nofollow noreferrer">this</a> or <a href="https://spectrum.ieee.org/computing/software/the-desperate-quest-for-genomic-compression-algorithms" rel="nofollow noreferrer">this</a>) discuss how genetic data will have scalability problems, given the huge file size of the human genome. The most straightforward encoding (see <a href="https://stackoverflow.com/questions/8954571/how-much-storage-would-be-required-to-store-a-human-genome">here</a>) of the human genome requires about 700 Mb. </p>

<p>I came across this <a href="https://academic.oup.com/bioinformatics/article/25/2/274/218156" rel="nofollow noreferrer">paper</a> claiming to be able to store the human genome in about 4 Mb, having a reference genome and exploiting the fact that all human genomes are mostly equal. This paper is much older than the the other references discussing the scalability problems. Why is this technique not widely used? </p>

CC BY-SA 4.0
2021-09-12T22:35:09.113Z
https://bioinformatics.stackexchange.com/questions/12824/theoretical-limit-of-human-genome-compression?noredirect=1&lq=1
Alex Reynolds
<phylogenetics><human-genome>
Compressing the human genome to few megabytes
# Compressing the human genome to few megabytes

(Asked by: username_0 on 2020-04-03)

Multiple sources (see for instance [this](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5958914/) or [this](https://spectrum.ieee.org/computing/software/the-desperate-quest-for-genomic-compression-algorithms)) discuss how genetic data will have scalability problems, given the huge file size of the human genome. The most straightforward encoding (see [here](https://stackoverflow.com/questions/8954571/how-much-storage-would-be-required-to-store-a-human-genome)) of the human genome requires about 700 Mb.

I came across this [paper](https://academic.oup.com/bioinformatics/article/25/2/274/218156) claiming to be able to store the human genome in about 4 Mb, having a reference genome and exploiting the fact that all human genomes are mostly equal. This paper is much older than the the other references discussing the scalability problems. Why is this technique not widely used?

**Tags:** phylogenetics, human-genome
> **Comments:**
> **username_1**: https://bioinformatics.stackexchange.com/questions/12824/theoretical-limit-of-human-genome-compression?noredirect=1&lq=1  

---

## 2 answers

### Answer by username_2 on 2021-07-14. Score: 3

*Answer from @alex-reynolds, converted from comments:*

As with any compression scheme, if you have to store a reference genome, along with the 4 Mb "diff", is it really compressed?

<http://mattmahoney.net/dc/dce.html>

For sequencing data, you might also look at the CRAM format, which also uses diffs against a reference genome to store reduced datasets. Labs are switching from BAM to CRAM to manage storage of increased numbers of sequencing datasets, with associated changes to pipelines and toolkits to use this format. For performance reasons, BAM may be preferred for analysis, and CRAM for long-term storage.

<https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats>

<https://www.ga4gh.org/news/cram-compression-for-genomics/>


---

### Answer by username_3 on 2021-08-13. Score: 1

I saw this paper a few weeks back that claims to offer 4x more compression than typical formats for both fastqs and BAM/CRAM files.

Genozip - A Universal Extensible Genomic Data Compressor

<https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077>


---

