<p><code>bwa mem</code> is newer, faster, and [should be] more accurate, particularly for longer reads.</p>

<p>From the <code>bwa</code> man page (presumably in Heng Li's own words):</p>

<blockquote>
  <p>BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms:  BWA-backtrack, BWA-SW and BWA-MEM. The first algorithm  is designed for Illumina sequence reads up to 100bp, while the rest two for longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar  features such as long-read support and split alignment, but <strong><em>BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and  more  accurate.</em></strong> BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.</p>
</blockquote>

CC BY-SA 3.0
2017-06-04T10:57:04.763Z
Yes, I know that's what he claims. That's just not what I've seen. Do you have any benchmarks or a review article or anything demonstrating that bwa-mem is indeed more accurate?
terdon
CC BY-SA 3.0
2017-06-04T11:05:12.133Z
Heng Li's paper on BWA is [here](https://arxiv.org/abs/1303.3997). It compares the accuracy of BWA-MEM with BWA-SW, and demonstrates that BWA-MEM has a higher number of mapped reads for the same number of mismapped reads (see Fig. 1).
gringer
CC BY-SA 3.0
2017-06-04T11:10:26.633Z
Yes, but he doesn't compare it to bwa backtrack which is what I am asking about.
terdon
CC BY-SA 3.0
2017-06-04T11:34:36.270Z
[Brian Bushnell](https://www.biostars.org/p/199625/#199627) seems to think that BWA-MEM is more accurate than BWA-backtrack.
gringer
CC BY-SA 3.0
2017-06-04T12:06:08.060Z
Yes, many people do. But I haven't seen anyone give evidence supporting this and my tests have shown it isn't.
terdon
CC BY-SA 3.0
2017-06-04T21:10:24.897Z
To be fair, you also haven't given any evidence supporting this. It would be great if you could add a graph, or other results into your question (preferably with a downloadable read/reference set).
gringer
CC BY-SA 3.0
2017-06-04T22:11:11.683Z
Yes indeed. I should show my data. To be honest, I did this more than a year ago and just don't have them anymore. It is absolutely possible that I'm wrong and there is no reason whatsoever for anyone to believe me just because I say so. I was hoping someone could answer with benchmarks of their own or, even better, with a peer-reviewed article showing comparison data.
terdon
CC BY-SA 3.0
2017-06-04T07:38:44.210Z
2017-06-04T07:38:44.210Z
gringer
<p>The industry standard for aligning short reads seems to be bwa-mem. However, in my tests I have seen that using bwa backtrack (bwa-aln + bwa-sampe + bwa-samse) performs better. It is slightly slower, but gives significantly better results in terms of both sensitivity and specificity. I have tested it using the genome in a bottle data and public samples (NA12878 and NA12877 among others) and found that backtrack consistently outperformed bwa-mem. </p>

<p>So why is bwa-mem the standard? Am I wrong and other tests have shown the opposite? I don't really see how since I tested using the most common datasets and validation data. Is it that the slight increase in efficiency outweighs the decrease in performance? </p>

<p>The only other explanation I can see is that bwa backtrack is designed specifically for Illumina reads and all my tests have been on Illumina data. Is it just that bwa-mem is "sequencer agnostic"? So that we can use the same algorithm irrespective of what sequencing platform is used? In that case, it makes sense to use backtrack for if we only deal with Illumina data and mem if we can have different sequencers. But, if so, seeing as Illumina is so widespread, why isn't backtrack used more often on Illumina data? I feel I must be missing something.</p>

CC BY-SA 3.0
2017-06-03T14:03:44.117Z
Heng Li would be the best person to respond to this....

Also it might be worth citing existing benchmarks or giving details of those which you have run.
Matt Bashton
CC BY-SA 3.0
2017-06-03T14:06:11.030Z
There is always the bwa mem paper too https://arxiv.org/pdf/1303.3997.pdf
Matt Bashton
CC BY-SA 3.0
2017-06-03T14:06:15.793Z
@MatthewBashton yes, I know. Unfortunately, I don't have them in a form that's easy to show and I ran them more than a year ago now so they might not even be relevant anymore. I am hoping that others have seen similar issues and know why one would be preferred over the other. Or that a review article has been published comparing them and my pubmed-fu just failed me.  If not, I guess I'll rerun them myself, look into it more deeply and post a new question.
terdon
CC BY-SA 3.0
2017-06-03T17:54:59.073Z
What's your read length? http://bio-bwa.sourceforge.net recommends mem as best for >70 bp. It would be interesting to see a reproducible benchmark if you've found otherwise
blmoore
CC BY-SA 3.0
2017-06-03T17:56:32.873Z
@blmoore it's mostly ~150nt on average. Which is precisely why I was surprised to see this. However, I realize that unless I show my data (which I will have to regenerate, I am afraid) it is hard for you to just take my word for it. I was kind of hoping this was a well known issue and I was just ignorant.
terdon
CC BY-SA 3.0
2017-06-03T18:30:31.343Z
Hi @terdon; our sequencing core team has looked at BWA-MEM vs backtrack for several data sets and have consistently found that MEM is both faster and more accurate than backtrack. We've also found that if you have significant quality drop-offs at the tail backtrack's performance suffers heavily... and that's another advantage of MEM: you don't need quality trimming, where backtrack needs reads to be mapped in full-length, which we have observed independently of, and in addition to Heng Li (see [this thread](https://www.biostars.org/p/90149/)).
Wyatt McDonnell
CC BY-SA 3.0
2017-06-03T18:32:19.303Z
@WyattMcDonnell thank you, that looks interesting although I have trouble reading that kind of chaotic thread. Do you think you could whip up an answer with these points?
terdon
CC BY-SA 4.0
2018-12-06T22:42:06.607Z
our department uses bwa backtrack for illumina sequencing data as well. as a side note, minimap2 (written by heng li) outperforms bwa mem for long reads, so I am having trouble figuring out where bwa mem falls into the picture.
d_kennetz
CC BY-SA 4.0
2019-06-27T10:18:54.457Z
Closely related (arguably a duplicate): https://bioinformatics.stackexchange.com/q/15/3144. Still no empirical comparison of the effectiveness of the two approaches of the sort that the OP here was asking for, though.
Mark Amery
CC BY-SA 4.0
2019-07-01T19:20:08.430Z
Possible duplicate of [Difference between BWA-backtrack and BWA-MEM](https://bioinformatics.stackexchange.com/questions/15/difference-between-bwa-backtrack-and-bwa-mem)
Bioathlete
<ngs><read-mapping><bwa>
Why is bwa-mem the standard algorithm when using bwa?
# Why is bwa-mem the standard algorithm when using bwa?

(Asked by: username_0 on 2017-06-03)

The industry standard for aligning short reads seems to be bwa-mem. However, in my tests I have seen that using bwa backtrack (bwa-aln + bwa-sampe + bwa-samse) performs better. It is slightly slower, but gives significantly better results in terms of both sensitivity and specificity. I have tested it using the genome in a bottle data and public samples (NA12878 and NA12877 among others) and found that backtrack consistently outperformed bwa-mem.

So why is bwa-mem the standard? Am I wrong and other tests have shown the opposite? I don't really see how since I tested using the most common datasets and validation data. Is it that the slight increase in efficiency outweighs the decrease in performance?

The only other explanation I can see is that bwa backtrack is designed specifically for Illumina reads and all my tests have been on Illumina data. Is it just that bwa-mem is "sequencer agnostic"? So that we can use the same algorithm irrespective of what sequencing platform is used? In that case, it makes sense to use backtrack for if we only deal with Illumina data and mem if we can have different sequencers. But, if so, seeing as Illumina is so widespread, why isn't backtrack used more often on Illumina data? I feel I must be missing something.

**Tags:** ngs, read-mapping, bwa
> **Comments:**
> **username_1**: Heng Li would be the best person to respond to this....  Also it might be worth citing existing benchmarks or giving details of those which you have run.  
> **username_1**: There is always the bwa mem paper too https://arxiv.org/pdf/1303.3997.pdf  
> **username_0**: @MatthewBashton yes, I know. Unfortunately, I don't have them in a form that's easy to show and I ran them more than a year ago now so they might not even be relevant anymore. I am hoping that others have seen similar issues and know why one would be preferred over the other. Or that a review article has been published comparing them and my pubmed-fu just failed me.  If not, I guess I'll rerun them myself, look into it more deeply and post a new question.  
> **username_2**: What's your read length? http://bio-bwa.sourceforge.net recommends mem as best for >70 bp. It would be interesting to see a reproducible benchmark if you've found otherwise  
> **username_0**: @blmoore it's mostly ~150nt on average. Which is precisely why I was surprised to see this. However, I realize that unless I show my data (which I will have to regenerate, I am afraid) it is hard for you to just take my word for it. I was kind of hoping this was a well known issue and I was just ignorant.  
> **username_3**: Hi @terdon; our sequencing core team has looked at BWA-MEM vs backtrack for several data sets and have consistently found that MEM is both faster and more accurate than backtrack. We've also found that if you have significant quality drop-offs at the tail backtrack's performance suffers heavily... and that's another advantage of MEM: you don't need quality trimming, where backtrack needs reads to be mapped in full-length, which we have observed independently of, and in addition to Heng Li (see [this thread](https://www.biostars.org/p/90149/)).  
> **username_0**: @WyattMcDonnell thank you, that looks interesting although I have trouble reading that kind of chaotic thread. Do you think you could whip up an answer with these points?  
> **username_4**: our department uses bwa backtrack for illumina sequencing data as well. as a side note, minimap2 (written by heng li) outperforms bwa mem for long reads, so I am having trouble figuring out where bwa mem falls into the picture.  
> **username_5**: Closely related (arguably a duplicate): https://bioinformatics.stackexchange.com/q/15/3144. Still no empirical comparison of the effectiveness of the two approaches of the sort that the OP here was asking for, though.  
> **username_6**: Possible duplicate of [Difference between BWA-backtrack and BWA-MEM](https://bioinformatics.stackexchange.com/questions/15/difference-between-bwa-backtrack-and-bwa-mem)  

---

## 1 answers

### Answer by username_7 on 2017-06-04. Score: 7 | Accepted answer

`bwa mem` is newer, faster, and [should be] more accurate, particularly for longer reads.

From the `bwa` man page (presumably in Heng Li's own words):

> BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM. The first algorithm is designed for Illumina sequence reads up to 100bp, while the rest two for longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar features such as long-read support and split alignment, but ***BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and more accurate.*** BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.

> **Comments:**
> **username_0**: Yes, I know that's what he claims. That's just not what I've seen. Do you have any benchmarks or a review article or anything demonstrating that bwa-mem is indeed more accurate?  
> **username_7**: Heng Li's paper on BWA is [here](https://arxiv.org/abs/1303.3997). It compares the accuracy of BWA-MEM with BWA-SW, and demonstrates that BWA-MEM has a higher number of mapped reads for the same number of mismapped reads (see Fig. 1).  
> **username_0**: Yes, but he doesn't compare it to bwa backtrack which is what I am asking about.  
> **username_7**: [Brian Bushnell](https://www.biostars.org/p/199625/#199627) seems to think that BWA-MEM is more accurate than BWA-backtrack.  
> **username_0**: Yes, many people do. But I haven't seen anyone give evidence supporting this and my tests have shown it isn't.  
> **username_7**: To be fair, you also haven't given any evidence supporting this. It would be great if you could add a graph, or other results into your question (preferably with a downloadable read/reference set).  
> **username_0**: Yes indeed. I should show my data. To be honest, I did this more than a year ago and just don't have them anymore. It is absolutely possible that I'm wrong and there is no reason whatsoever for anyone to believe me just because I say so. I was hoping someone could answer with benchmarks of their own or, even better, with a peer-reviewed article showing comparison data.  

---

