<p>That can be easily done with MiXCR software.
If you first analyze the raw data using mixcr, which can be easily done with <code>mixcr analyze</code> command depending on your data structure:</p>
<p>e.g.</p>
<pre><code>mixcr analyze generic-tcr-amplicon \
--species mmu \
--rna \
--rigid-left-alignment-boundary \
--floating-right-alignment-boundary C \
  input_R1.fastq.gz \
  input_R2.fastq.gz \
  result
</code></pre>
<p>More on that here:</p>
<p><a href="https://docs.milaboratories.com/mixcr/reference/mixcr-analyze/" rel="nofollow noreferrer">https://docs.milaboratories.com/mixcr/reference/mixcr-analyze/</a></p>
<p>After that you will have .clns file for every sample.</p>
<p>Then, to merge all samples you can use <code>mixcr exportClonesOverlap</code> and all .clns files as an input, e.g.:</p>
<pre><code>mixcr exportClonesOverlap \
  --criteria &quot;CDR3|AA|V|J&quot; \
  clonesets/*.clns
  overlapTable.tsv 
</code></pre>
<p><code>--criteria</code> allows you to specify the criteria required for clonotypes to be treated as identical. <code>&quot;CDR3|AA|V|J&quot;</code> means that two clonotypes have to share cdr3 amino acid sequence, v gene and j gene.</p>
<p>overlapTable.tsv is exactly what you need.</p>
<p>There are other parameters you can tune (like export only productive clonotypes etc.), more on that here:</p>
<p><a href="https://docs.milaboratories.com/mixcr/reference/mixcr-exportClonesOverlap/#example" rel="nofollow noreferrer">https://docs.milaboratories.com/mixcr/reference/mixcr-exportClonesOverlap/#example</a></p>

CC BY-SA 4.0
2023-01-24T17:09:32.177Z
Please avoid link only answers, they are useless if the link dies. Summarise the main points of the tutorial in the post. Thanks
user438383
CC BY-SA 4.0
2023-01-24T18:43:21.350Z
Also please disclose if you are in anyway involved with the software you are sharing.
user438383
CC BY-SA 4.0
2023-01-26T02:27:10.470Z
Perfect thank you for the comprehensive update. It is possible Mark Izraelson is employed by MiLaboratories, but we can't be certain.
M__
CC BY-SA 4.0
2023-01-30T09:01:24.487Z
Hi guys, apologies for initial post. I'm only starting to answer questions on Stackexchange, and will do my best to provide rigorous and useful answers for the community. And yes, I'm involved in MiXCR development for many years so far, so I'm first of all targeting to answer questions related to T and B cell NGS data processing.
Mark Izraelson
CC BY-SA 4.0
2023-02-17T12:21:05.620Z
@MarkIzraelson no worries, the rules here are a bit stricter than other internet spaces :) And not accusing you of any malfeasance, it's just part of the rules here to disclose affiliation. But it's great to have software developers here answering questions, so we welcome you to keep answering!
user438383
CC BY-SA 4.0
2023-01-24T16:22:24.970Z
2023-01-25T21:45:14.377Z
2023-01-25T21:45:14.377Z
Mark Izraelson
<p>You could give a try with dplyr's <code>full_join()</code> and purrr's <code>reduce()</code>. <code>full_join()</code> will make sure that you won't lose any data and <code>reduce()</code> will merge tables in your list from left to right, left being the first table of the list in the first iteration but would become merged <em>table1+table2</em> in the second iteration and so on. This left to right "reduction" will also be helpful when renaming the columns of the resulting table.</p>

<p>Without your workspace I have attempted to replicate your data: There is a list of 3 data frames, each with 2 columns:</p>

<pre><code>library(dplyr)
library(purrr)

df1 &lt;- data.frame(CDR3.aa = c("A", "B", "C"),
                  Count = c(10,20,100))

df2 &lt;- data.frame(CDR3.aa = c("A", "B", "D"),
                  Count = c(100,200,300))

df3 &lt;- data.frame(CDR3.aa = c("D", "E", "F"),
                  Count = c(10,20,100))

my_list &lt;- list(df1 = df1, df2 = df2, df3 = df3)

&gt; my_list
$df1
  CDR3.aa Count
1       A    10
2       B    20
3       C   100

$df2
  CDR3.aa Count
1       A   100
2       B   200
3       D   300

$df3
  CDR3.aa Count
1       D    10
2       E    20
3       F   100

# "by" argument specifies which column to use when merging
my_table &lt;- reduce(my_list, full_join, by = "CDR3.aa")

names(my_table) &lt;- c("CDR3.aa", names(my_list))

&gt; my_table
  CDR3.aa df1 df2 df3
1       A  10 100  NA
2       B  20 200  NA
3       C 100  NA  NA
4       D  NA 300  10
5       E  NA  NA  20
6       F  NA  NA 100
</code></pre>

<p>Clonotypes that are not present in table will be "NA".</p>

CC BY-SA 4.0
2019-08-12T13:02:14.347Z
Sounds like it could work.  I'll give it a try and report back.  Thanks.
Lou_A
CC BY-SA 4.0
2019-08-12T18:33:18.787Z
haci, it worked very well.  I just had to convert the initial samples to data.frames so I could run reduce on it.  It's a very clean elegant solution.  Thank you very much for your help.
Lou_A
CC BY-SA 4.0
2019-08-10T20:35:54.313Z
2019-08-10T20:40:57.357Z
2019-08-10T20:40:57.357Z
haci
<p>I’m trying to develop a table from a series of lists generated using the Immunarch library to process TCR sequencing results. Each list is comprised of CDR3.aa (clone) information which are character strings and their count in a particular sample. The clones are short but vary between 7 and 20 (or more).  Each list has a header that identifies the sample sequenced. I have a list of 66 samples. Each sample can have several thousand clone strings. Not every clone is contained in every sample so the number of clones listed in the samples varies.  Here’s an example of the structure of a single sample list.</p>

<pre><code>$PreAg_18_2
CDR3.aa           Count
CASSYGTAYTGELFF   1623
CASSRGDSDNSPLHF   1440
CASSREKAFF        1161
CSGMGALAKNIQYF     949
CSAYTGLSYEQYF      813
CASSLSLAVNSPLHF    634
CAIRDTPGSPQHF      574
CATGQVNTEAFF       555
CASSLKGQGGSPLHF    499
CASSYSRSPQPQHF     478
</code></pre>

<p>I want to combine the results in a single table showing the clone counts with all the clone strings listed on the y-axis and the sample ID listed on the x-axis. For example:  </p>

<pre><code>                10_pep_10_1     preAg_10_2      Dec_2_18_1  …... 
CASSYGTAYTGELFF    1623         234             0
CASSRGDSDNSPLHF    1440         522             28
CASSREKAFF         1161         445             50  
CSGMGALAKNIQYF      949         24              0
CASSYSRSPQPQHF      478         0               398
.
.
</code></pre>

<p>Currently, I'm working with R because that's what Immunarch is in, but I wonder if a) There's a better way to do this in Python, or b) If it's doable in r, then how do I generate the matrix matches.  The main problem is that not all the samples have the same number of clones, so I first have to separate each sample data before I can convert it to a data frame, and then how can I match the actual count with the actual clone.  I can't just merge the data frames.  Any suggestion is greatly appreciated.</p>

<p>Here's what i have for code right now.
Code:
    library(immunarch)
    library(stringr)
    library(plyr)</p>

<pre><code>immdata = repLoad("/mnt/data/Development/Analysis_Scripts/input_files/")

all &lt;- immdata$data

# Get list headers (names) and convert to df
sample.id &lt;- names(all)
sample.id &lt;- data.frame(sample.id)

# Get list of clones and filter for unique clones per list.
for (i in 1:length(all)){
    all[[i]]$Sample.ID&lt;-names(all)[i]
    all[[i]]&lt;-all[[i]][,c("CDR3.aa", "Clones")]
}

# all is the variable that contains the list of samples and clones.
all.split &lt;- split(all, )

# make vector of all clones
all.clones &lt;- unlist(all, use.names=FALSE)

# Removes clone repeats
all.clones.u &lt;- unique(all.clones)

# convert list of clones to data frame
all.clones.u &lt;- data.frame(all.clones.u)
</code></pre>

<p>At this point I have both the list of sample ids and the list of clones.  My problem is now matching each count with respective sample and  clone in the table.  Any suggestions?</p>

CC BY-SA 4.0
2019-08-10T13:58:13.147Z
It's hard to come up with a working code snippet if the workspace can't be recreated. Could you provide a minimal mock example of `all` that we can use? But based on what I think you want to do, I'd probably use a combination of `left_join` of the `dplyr` package combined with `Reduce`.
Sebastian Müller
CC BY-SA 4.0
2019-08-12T13:03:27.443Z
Thanks, Sebastian, I apologize for not providing a more clear picture of my current data, but I think you captured it correctly.  I'll give it a try.
Lou_A
<r>
Creating a count table using R
# Creating a count table using R

(Asked by: username_0 on 2019-08-09)

I’m trying to develop a table from a series of lists generated using the Immunarch library to process TCR sequencing results. Each list is comprised of CDR3.aa (clone) information which are character strings and their count in a particular sample. The clones are short but vary between 7 and 20 (or more).  Each list has a header that identifies the sample sequenced. I have a list of 66 samples. Each sample can have several thousand clone strings. Not every clone is contained in every sample so the number of clones listed in the samples varies.  Here’s an example of the structure of a single sample list.
$PreAg_18_2
CDR3.aa           Count
CASSYGTAYTGELFF   1623
CASSRGDSDNSPLHF   1440
CASSREKAFF        1161
CSGMGALAKNIQYF     949
CSAYTGLSYEQYF      813
CASSLSLAVNSPLHF    634
CAIRDTPGSPQHF      574
CATGQVNTEAFF       555
CASSLKGQGGSPLHF    499
CASSYSRSPQPQHF     478

I want to combine the results in a single table showing the clone counts with all the clone strings listed on the y-axis and the sample ID listed on the x-axis. For example:  
                10_pep_10_1     preAg_10_2      Dec_2_18_1  …... 
CASSYGTAYTGELFF    1623         234             0
CASSRGDSDNSPLHF    1440         522             28
CASSREKAFF         1161         445             50  
CSGMGALAKNIQYF      949         24              0
CASSYSRSPQPQHF      478         0               398
.
.

Currently, I'm working with R because that's what Immunarch is in, but I wonder if a) There's a better way to do this in Python, or b) If it's doable in r, then how do I generate the matrix matches.  The main problem is that not all the samples have the same number of clones, so I first have to separate each sample data before I can convert it to a data frame, and then how can I match the actual count with the actual clone.  I can't just merge the data frames.  Any suggestion is greatly appreciated.
Here's what i have for code right now.
Code:
    library(immunarch)
    library(stringr)
    library(plyr)
immdata = repLoad("/mnt/data/Development/Analysis_Scripts/input_files/")

all <- immdata$data

# Get list headers (names) and convert to df
sample.id <- names(all)
sample.id <- data.frame(sample.id)

# Get list of clones and filter for unique clones per list.
for (i in 1:length(all)){
    all[[i]]$Sample.ID<-names(all)[i]
    all[[i]]<-all[[i]][,c("CDR3.aa", "Clones")]
}

# all is the variable that contains the list of samples and clones.
all.split <- split(all, )

# make vector of all clones
all.clones <- unlist(all, use.names=FALSE)

# Removes clone repeats
all.clones.u <- unique(all.clones)

# convert list of clones to data frame
all.clones.u <- data.frame(all.clones.u)

At this point I have both the list of sample ids and the list of clones.  My problem is now matching each count with respective sample and  clone in the table.  Any suggestions?

**Tags:** r
> **Comments:**
> **username_1**: It's hard to come up with a working code snippet if the workspace can't be recreated. Could you provide a minimal mock example of `all` that we can use? But based on what I think you want to do, I'd probably use a combination of `left_join` of the `dplyr` package combined with `Reduce`.  
> **username_0**: Thanks, Sebastian, I apologize for not providing a more clear picture of my current data, but I think you captured it correctly.  I'll give it a try.  

---

## 2 answers

### Answer by username_5 on 2019-08-10. Score: 4 | Accepted answer

You could give a try with dplyr's full_join() and purrr's reduce(). full_join() will make sure that you won't lose any data and reduce() will merge tables in your list from left to right, left being the first table of the list in the first iteration but would become merged table1+table2 in the second iteration and so on. This left to right "reduction" will also be helpful when renaming the columns of the resulting table.
Without your workspace I have attempted to replicate your data: There is a list of 3 data frames, each with 2 columns:
library(dplyr)
library(purrr)

df1 <- data.frame(CDR3.aa = c("A", "B", "C"),
                  Count = c(10,20,100))

df2 <- data.frame(CDR3.aa = c("A", "B", "D"),
                  Count = c(100,200,300))

df3 <- data.frame(CDR3.aa = c("D", "E", "F"),
                  Count = c(10,20,100))

my_list <- list(df1 = df1, df2 = df2, df3 = df3)

> my_list
$df1
  CDR3.aa Count
1       A    10
2       B    20
3       C   100

$df2
  CDR3.aa Count
1       A   100
2       B   200
3       D   300

$df3
  CDR3.aa Count
1       D    10
2       E    20
3       F   100

# "by" argument specifies which column to use when merging
my_table <- reduce(my_list, full_join, by = "CDR3.aa")

names(my_table) <- c("CDR3.aa", names(my_list))

> my_table
  CDR3.aa df1 df2 df3
1       A  10 100  NA
2       B  20 200  NA
3       C 100  NA  NA
4       D  NA 300  10
5       E  NA  NA  20
6       F  NA  NA 100

Clonotypes that are not present in table will be "NA".

> **Comments:**
> **username_0**: Sounds like it could work.  I'll give it a try and report back.  Thanks.  
> **username_0**: haci, it worked very well.  I just had to convert the initial samples to data.frames so I could run reduce on it.  It's a very clean elegant solution.  Thank you very much for your help.  

---

### Answer by username_2 on 2023-01-24. Score: 1

That can be easily done with MiXCR software.
If you first analyze the raw data using mixcr, which can be easily done with mixcr analyze command depending on your data structure:
e.g.
mixcr analyze generic-tcr-amplicon \
--species mmu \
--rna \
--rigid-left-alignment-boundary \
--floating-right-alignment-boundary C \
  input_R1.fastq.gz \
  input_R2.fastq.gz \
  result

More on that here:
https://docs.milaboratories.com/mixcr/reference/mixcr-analyze/
After that you will have .clns file for every sample.
Then, to merge all samples you can use mixcr exportClonesOverlap and all .clns files as an input, e.g.:
mixcr exportClonesOverlap \
  --criteria "CDR3|AA|V|J" \
  clonesets/*.clns
  overlapTable.tsv 

--criteria allows you to specify the criteria required for clonotypes to be treated as identical. "CDR3|AA|V|J" means that two clonotypes have to share cdr3 amino acid sequence, v gene and j gene.
overlapTable.tsv is exactly what you need.
There are other parameters you can tune (like export only productive clonotypes etc.), more on that here:
https://docs.milaboratories.com/mixcr/reference/mixcr-exportClonesOverlap/#example

> **Comments:**
> **username_3**: Please avoid link only answers, they are useless if the link dies. Summarise the main points of the tutorial in the post. Thanks  
> **username_3**: Also please disclose if you are in anyway involved with the software you are sharing.  
> **username_4**: Perfect thank you for the comprehensive update. It is possible Mark Izraelson is employed by MiLaboratories, but we can't be certain.  
> **username_2**: Hi guys, apologies for initial post. I'm only starting to answer questions on Stackexchange, and will do my best to provide rigorous and useful answers for the community. And yes, I'm involved in MiXCR development for many years so far, so I'm first of all targeting to answer questions related to T and B cell NGS data processing.  
> **username_3**: @MarkIzraelson no worries, the rules here are a bit stricter than other internet spaces :) And not accusing you of any malfeasance, it's just part of the rules here to disclose affiliation. But it's great to have software developers here answering questions, so we welcome you to keep answering!  

---

