<p>I'm late to the party but simply run <code>cellranger mkfastq</code> twice, with different arguments for the mask and with <code>--filter-dual-index</code> for the double-indexed samples. In a second step you then have to disentangle the dual A+B reads from single-index A reads based on the readID's that are found in both of the demultiplexings. This is cumbersome and time+diskspace consuming, but it can be done. I would use some Unix tools for that but there are probably smarter ways.</p>
<p>EDIT: I mistakenly said <code>bcl2fastq</code>, this must be <code>cellranger mkfastq</code>. (thanks AmadeusDrZaius)</p>

CC BY-SA 4.0
2021-05-25T18:46:23.320Z
Thanks for your suggestion. I only see that "--filter-dual-index" option available for `cellranger mkfastq`, rather than bcl2fastq itself. Is that where you saw the option?
AmadeusDrZaius
CC BY-SA 4.0
2021-05-26T19:54:49.303Z
ah yes, that is true, sorry. I guess you would have to do an additional filtering step on the FASTQ file, the second word on the ID lines is 'index1+index2' for properly demultiplexed reads.
plijnzaad
CC BY-SA 4.0
2021-05-21T08:57:08.707Z
2021-05-26T19:58:00.250Z
2021-05-26T19:58:00.250Z
plijnzaad
<p>Have you just tried giving bcl2fastq one sample sheet with a mix of single and dual indices?  I don't think what you are trying to do is a problem.</p>

<blockquote>
  <p>if any single index A is part of a dual index A+B,</p>
</blockquote>

<p>Well, it might be a problem if you did that.  That was poor planning.</p>

CC BY-SA 4.0
2019-07-30T18:01:30.210Z
That was the first thing tried but unfortunately bcl2fastq will give the error "Barcodes have an unequal number of components" when you try to do that (using v2.17). I'll add a note to my question.
AmadeusDrZaius
CC BY-SA 4.0
2019-07-30T19:19:04.603Z
Are you sure that the single index samples don't have a site for the second index to sit down and read?  My lab does some samples with single indicies and some with double, but they always prep with libraries with kits meant for dual indexing, so the second index is always there, even if I ignore it.
swbarnes2
CC BY-SA 4.0
2019-07-30T19:33:06.290Z
That's a good idea that might be a good workaround in some cases, but I think the general answer is that they are not always physically dual-indexed. The use of "N" as described in the bcl2fastq documentation would work if not for the fact that a bug in bcl2fastq causes it to treat "N" as a literal base "N" instead of as a wildcard. Given that, I'm guessing that the Picard tools are more likely to have a full solution for this particular situation.
AmadeusDrZaius
CC BY-SA 4.0
2019-07-30T19:51:44.690Z
I don't think it's going to be trivial for software to figure out the difference between "second index doesn't match what I expect" and "second index isn't really here".  You might have to have bcl2fastq export the indices as fastqs, so you can use the quality score to parse yourself.
swbarnes2
CC BY-SA 4.0
2019-07-30T20:10:26.517Z
That's an interesting point. I expected that it would just make a "best guess", but I can see why it wouldn't. It sounds like that would primarily only be an issue in the case of "A" and "A+B" that I described above. If the case of "A" and "A+B" is avoided, it should be able to tell the difference. Thanks for your thoughts.
AmadeusDrZaius
CC BY-SA 4.0
2019-07-30T20:37:40.770Z
I would be surprised if the Experiment Manager software would allow a run configuration that was un-processable.  I take it the instrument was run without a sample sheet made by Experiment Manager?
swbarnes2
CC BY-SA 4.0
2019-07-30T16:15:38.150Z
2019-07-30T16:15:38.150Z
swbarnes2
<h2>The problem</h2>

<p>If I have a sample sheet that contains both single-indexed and dual-indexed samples, I can split it up into two sample sheets and then run bcl2fastq on each one. However, when doing this, large Undetermined fastq files are generated. E.g., when processing the single-indexed samples, all the dual-indexed samples go to Undetermined. And when processing the dual-indexed samples, the single-indexed samples go to Undetermined.</p>

<p>Additionally, because they are being processed separately, if any single index A is part of a dual index A+B, then when processing the single indexes, an A+B may be mistaken for an A, so it would seem that they need to processed simultaneously to avoid this mis-assignment.</p>

<h2>The question</h2>

<p>Given a sample sheet and directories of BCL files, how can such a set of sequencing data be demultiplexed correctly, either using bcl2fastq or the Picard tools?</p>

<p>To put it another way, I want to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. It can be assumed that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.</p>

<h2>Attempts at a solution</h2>

<h3>Using bcl2fastq directly</h3>

<p>If a sample sheet of a mix of indexes is given to bcl2fastq (v2.17) directly, it produces the error</p>

<pre><code>ERROR: bcl2fastq::common::Exception: Success (0): .../bcl2fastq2/src/cxx/lib/layout/BarcodeCollisionDetector.cpp(127): 
Throw in function void bcl2fastq::layout::BarcodeCollisionDetector::validateNewBarcodeSizesAgainstExisting(const std::vector&lt;long unsigned int&gt;&amp;) const
Dynamic exception type: boost::exception_detail::clone_impl&lt;bcl2fastq::layout::BarcodeCollisionError&gt;
std::exception::what: Barcodes have an unequal number of components.
</code></pre>

<blockquote>
  <p>Barcodes have an unequal number of components.</p>
</blockquote>

<h3>Using picard</h3>

<p>It seems that it should be possible using the Picard tools, but I have not found a way to set up the inputs to ExtractIlluminaBarcodes and IlluminaBasecallsToFastq that processes this configuration correctly.</p>

<p>The syntax for supporting multiple index configurations is not entirely clear. But when using various combinations of N and '*' on multiplex_params.tsv and barcodes.txt required by the Picard tools, a large Undetermined file is still produced and actual sample fastq files are tiny, indicating it is not processing them correctly.</p>

<h3>Using multiple calls to bcl2fastq</h3>

<p>As indicated in the discussion above, this suffers from the problem of having all the dual indexed samples going to the "Undetermined" file when processing the single-indexed samples, or vice versa, and creates two output directories of reports and stats, which must be merged.</p>

<h3>Padding empty indexes with N</h3>

<p>By padding, I mean converting ACGTACGT to ACGTACGT+NNNNNNNN so that the single-index samples in the sample sheet are "dual" as well. This strategy seems that it would work except there is a bug in bcl2fastq that it treats "N" literally instead of as a wildcard. <a href="https://support.illumina.com/content/dam/illumina-support/documents/downloads/software/bcl2fastq/bcl2fastq2-2-20-0-release-notes-1000000039216-00.pdf" rel="nofollow noreferrer">See the release notes for details.</a></p>

CC BY-SA 4.0
2019-07-30T15:27:48.643Z
Welcome to the site. Sorry but I don't understand which files do you have. Could you clarify what are your files? Or do you have a mix of single-indexed and dual-indexed and you can't differentiate between them?
llrs
CC BY-SA 4.0
2019-07-30T15:33:11.050Z
@llrs Hi, thanks for the question. In this case, I'm trying to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. You can assume that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.
AmadeusDrZaius
CC BY-SA 4.0
2019-07-30T15:34:32.993Z
Ok, now I understood the question, but I'm sorry I can't help with it. Perhaps you could [edit] your last comment in the question for other users.
llrs
<illumina><picard><demultiplexing><indexing>
How to demultiplex a mix of single-indexed and dual-indexed samples
# How to demultiplex a mix of single-indexed and dual-indexed samples

(Asked by: username_0 on 2019-07-30)

The problem
If I have a sample sheet that contains both single-indexed and dual-indexed samples, I can split it up into two sample sheets and then run bcl2fastq on each one. However, when doing this, large Undetermined fastq files are generated. E.g., when processing the single-indexed samples, all the dual-indexed samples go to Undetermined. And when processing the dual-indexed samples, the single-indexed samples go to Undetermined.
Additionally, because they are being processed separately, if any single index A is part of a dual index A+B, then when processing the single indexes, an A+B may be mistaken for an A, so it would seem that they need to processed simultaneously to avoid this mis-assignment.
The question
Given a sample sheet and directories of BCL files, how can such a set of sequencing data be demultiplexed correctly, either using bcl2fastq or the Picard tools?
To put it another way, I want to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. It can be assumed that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.
Attempts at a solution
Using bcl2fastq directly
If a sample sheet of a mix of indexes is given to bcl2fastq (v2.17) directly, it produces the error
ERROR: bcl2fastq::common::Exception: Success (0): .../bcl2fastq2/src/cxx/lib/layout/BarcodeCollisionDetector.cpp(127): 
Throw in function void bcl2fastq::layout::BarcodeCollisionDetector::validateNewBarcodeSizesAgainstExisting(const std::vector&) const
Dynamic exception type: boost::exception_detail::clone_impl
std::exception::what: Barcodes have an unequal number of components.


Barcodes have an unequal number of components.

Using picard
It seems that it should be possible using the Picard tools, but I have not found a way to set up the inputs to ExtractIlluminaBarcodes and IlluminaBasecallsToFastq that processes this configuration correctly.
The syntax for supporting multiple index configurations is not entirely clear. But when using various combinations of N and '*' on multiplex_params.tsv and barcodes.txt required by the Picard tools, a large Undetermined file is still produced and actual sample fastq files are tiny, indicating it is not processing them correctly.
Using multiple calls to bcl2fastq
As indicated in the discussion above, this suffers from the problem of having all the dual indexed samples going to the "Undetermined" file when processing the single-indexed samples, or vice versa, and creates two output directories of reports and stats, which must be merged.
Padding empty indexes with N
By padding, I mean converting ACGTACGT to ACGTACGT+NNNNNNNN so that the single-index samples in the sample sheet are "dual" as well. This strategy seems that it would work except there is a bug in bcl2fastq that it treats "N" literally instead of as a wildcard. See the release notes for details.

**Tags:** illumina, picard, demultiplexing, indexing
> **Comments:**
> **username_1**: Welcome to the site. Sorry but I don't understand which files do you have. Could you clarify what are your files? Or do you have a mix of single-indexed and dual-indexed and you can't differentiate between them?  
> **username_0**: @llrs Hi, thanks for the question. In this case, I'm trying to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. You can assume that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.  
> **username_1**: Ok, now I understood the question, but I'm sorry I can't help with it. Perhaps you could [edit] your last comment in the question for other users.  

---

## 2 answers

### Answer by username_2 on 2021-05-21. Score: 1

I'm late to the party but simply run `cellranger mkfastq` twice, with different arguments for the mask and with `--filter-dual-index` for the double-indexed samples. In a second step you then have to disentangle the dual A+B reads from single-index A reads based on the readID's that are found in both of the demultiplexings. This is cumbersome and time+diskspace consuming, but it can be done. I would use some Unix tools for that but there are probably smarter ways.

EDIT: I mistakenly said `bcl2fastq`, this must be `cellranger mkfastq`. (thanks AmadeusDrZaius)

> **Comments:**
> **username_0**: Thanks for your suggestion. I only see that "--filter-dual-index" option available for `cellranger mkfastq`, rather than bcl2fastq itself. Is that where you saw the option?  
> **username_2**: ah yes, that is true, sorry. I guess you would have to do an additional filtering step on the FASTQ file, the second word on the ID lines is 'index1+index2' for properly demultiplexed reads.  

---

### Answer by username_3 on 2019-07-30. Score: 0

Have you just tried giving bcl2fastq one sample sheet with a mix of single and dual indices? I don't think what you are trying to do is a problem.

> if any single index A is part of a dual index A+B,

Well, it might be a problem if you did that. That was poor planning.

> **Comments:**
> **username_0**: That was the first thing tried but unfortunately bcl2fastq will give the error "Barcodes have an unequal number of components" when you try to do that (using v2.17). I'll add a note to my question.  
> **username_3**: Are you sure that the single index samples don't have a site for the second index to sit down and read?  My lab does some samples with single indicies and some with double, but they always prep with libraries with kits meant for dual indexing, so the second index is always there, even if I ignore it.  
> **username_0**: That's a good idea that might be a good workaround in some cases, but I think the general answer is that they are not always physically dual-indexed. The use of "N" as described in the bcl2fastq documentation would work if not for the fact that a bug in bcl2fastq causes it to treat "N" as a literal base "N" instead of as a wildcard. Given that, I'm guessing that the Picard tools are more likely to have a full solution for this particular situation.  
> **username_3**: I don't think it's going to be trivial for software to figure out the difference between "second index doesn't match what I expect" and "second index isn't really here".  You might have to have bcl2fastq export the indices as fastqs, so you can use the quality score to parse yourself.  
> **username_0**: That's an interesting point. I expected that it would just make a "best guess", but I can see why it wouldn't. It sounds like that would primarily only be an issue in the case of "A" and "A+B" that I described above. If the case of "A" and "A+B" is avoided, it should be able to tell the difference. Thanks for your thoughts.  
> **username_3**: I would be surprised if the Experiment Manager software would allow a run configuration that was un-processable.  I take it the instrument was run without a sample sheet made by Experiment Manager?  

---

