<p>Just to clarify, when I asked you to share some of your files, I meant to share them in a way that I can reproduce your issue from start to finish. You read from at least four different files in a quadruple for-loop and I'm pretty sure your goal could be achieved much more straightforwardly than your current implementation.</p>
<p>Without a reproducible example, I can offer a best-guess on what's happening: you create <code>date_df</code> and <code>location_df</code>, but after you instantiate them, you never do anything with the objects. Your final write happens from <code>df</code>, so you'll need to add those columns, e.g.:</p>
<pre class="lang-py prettyprint-override"><code>df[&quot;Date&quot;] = date_df[&quot;Date&quot;][0]
df[&quot;Location&quot;] = location_df[&quot;Location&quot;][0]
</code></pre>
<p>The above is just an example, you'll need to make sure the <em>correct</em> value gets assigned to your <code>df</code> object instead of simply the first one.</p>

CC BY-SA 4.0
2020-10-31T23:22:08.403Z
Again, apologies I was late running for work and rather skimmed the post. So would this be implemented before the mutation_set loop or between it? Or before the df[mutation_header]? I assume the latter
Theo Jones
CC BY-SA 4.0
2020-11-01T01:42:45.920Z
I assumed the latter and it’s worked perfectly? Thanks again!
Theo Jones
CC BY-SA 4.0
2020-10-31T19:56:09.070Z
2020-10-31T19:56:09.070Z
Bastian Schiffthaler
<p>I'm trying to reuse some code I have previously used to add extra parameters.
In the initial code, it was used to give a 1/0 counter if a specific mutation was present (first code below)</p>
<pre><code>import re
import os
import pandas as pd
import logging
import sys
import csv

root = &quot;C:&quot;

os.chdir(root)

lineages = os.listdir('Lineages')
mutations = os.listdir('Mutations')

for lineage in lineages:
    
    df = pd.read_csv('Lineages/' + lineage).dropna()
    df.columns = [&quot;lineage&quot;]
    df['lineage'] = [x.split('|')[0].split('_')[-1]
                     for x in df['lineage']]
               

    for mutation in mutations:
        
        mutation_df = pd.read_csv('Mutations/' + mutation).dropna()
        mutation_header = mutation.replace(&quot;.txt&quot;, &quot;&quot;)
        mutation_df.columns = [mutation_header]
        mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] 
                          for x in mutation_df[mutation_header]]
        
        mutation_set = set(mutation_df[mutation_header])
        
        df[mutation_header] = [1 if x in mutation_set else 0
                               for x in df['lineage']]
        
    df.to_csv('Results9/' + lineage.replace(&quot;.txt&quot;, &quot;&quot;) + '.csv',
                 header = True, 
                 index = False)
</code></pre>
<p>However, I now wish to add extra parameters such as date of sample and location of sample.
The first code produces this type of counter system:</p>
<pre><code>lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A
417941,0,1,0,0,0,0,0,0,0,0,0
417942,0,1,0,0,0,0,0,0,0,0,0
417944,0,1,0,0,0,0,0,0,0,0,0
417946,0,1,0,0,0,0,0,0,0,0,0
417947,0,1,0,0,0,0,0,0,0,0,0
417948,0,1,0,0,0,0,0,0,0,0,0
417950,0,0,0,0,0,0,0,0,0,0,0
417955,0,1,0,0,0,0,0,0,0,0,0
431011,0,1,0,0,0,0,0,0,0,0,0
</code></pre>
<p>This code is fine and working, but if I try and add the second parameter (date/location) with the following code:</p>
<pre><code>import re
import os
import pandas as pd
import logging
import sys
import csv

root = &quot;C:/Users/jones/Downloads/SARSCOV2/&quot;

os.chdir(root)

lineages = os.listdir('Lineages')
mutations = os.listdir('Mutations')
dates = os.listdir('Dates')
locations = os.listdir('Locations')

for date in dates:
            
    date_df = pd.read_csv('Dates/' + date).dropna()
    date_df.columns = [&quot;Date&quot;]
    date_df['Date'] = [x.split('|')[-1].split('/')[-1] 
                       for x in date_df['Date']]
            
    for location in locations:
                
        location_df = pd.read_csv('Locations/' + location).dropna()
        location_df.columns = [&quot;Location&quot;]
        location_df['Location'] = [x.split('/')[1].split('_')[0]
                                   for x in location_df['Location']]
            
        for lineage in lineages:
    
            df = pd.read_csv('Lineages/' + lineage).dropna()
            df.columns = [&quot;lineage&quot;]
            df['lineage'] = [x.split('_')[-1].split('|')[0]
                             for x in df['lineage']]
               

            for mutation in mutations:
        
                mutation_df = pd.read_csv('Mutations/' + mutation).dropna()
                mutation_header = mutation.replace(&quot;.txt&quot;, &quot;&quot;)
                mutation_df.columns = [mutation_header]
                mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] 
                                                for x in mutation_df[mutation_header]]
        
                mutation_set = set(mutation_df[mutation_header])
        
                df[mutation_header] = [1 if x in mutation_set else 0
                                       for x in df['lineage']]
   
  
            df.to_csv('Results_3/' + lineage.replace(&quot;.txt&quot;, &quot;&quot;) + '.csv',
                 header = True, 
                 index = False)
</code></pre>
<p>I retrieve this data, where the counting is maintaibned, but the new parameters (DATE/LOCATION) are not added</p>
<pre><code>lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A
417941,0,1,0,0,0,0,0,0,0,0,0
417942,0,1,0,0,0,0,0,0,0,0,0
417944,0,1,0,0,0,0,0,0,0,0,0
417946,0,1,0,0,0,0,0,0,0,0,0
417947,0,1,0,0,0,0,0,0,0,0,0
417948,0,1,0,0,0,0,0,0,0,0,0
417950,0,0,0,0,0,0,0,0,0,0,0
417955,0,1,0,0,0,0,0,0,0,0,0
431011,0,1,0,0,0,0,0,0,0,0,0
</code></pre>
<p>In short, my end goal is to:</p>
<ul>
<li>Add Lineage ID|Location of Sample|DD/MM/YYY|Mutations| -- as the headers</li>
<li>still maintain the mutation counter</li>
<li>Use the mutation counter as a guide to determine where and when the genotypes (mutation combinations) arose.</li>
</ul>
<p>The files where I am retrieving the parameters looks like this also:</p>
<pre><code>
hCoV-19/Singapore/4/2020|EPI_ISL_410535|2020-02-03

hCoV-19/USA/WA13-UW9/2020|EPI_ISL_413601|2020-03-02

hCoV-19/USA/WA-UW142/2020|EPI_ISL_416680|2020-03-11

hCoV-19/USA/WA-UW143/2020|EPI_ISL_416681|2020-03-11

hCoV-19/USA/WA-UW144/2020|EPI_ISL_416682|2020-03-09

hCoV-19/USA/WA-UW145/2020|EPI_ISL_416683|2020-03-15

hCoV-19/USA/WA-UW146/2020|EPI_ISL_416684|2020-03-14

hCoV-19/USA/WA-UW147/2020|EPI_ISL_416685|2020-03-15

hCoV-19/USA/WA-UW148/2020|EPI_ISL_416686|2020-03-14
</code></pre>
<p>A very laborious, and poorly explained question; I apologise in advance!</p>

CC BY-SA 4.0
2020-10-30T12:30:53.733Z
I have realised an error in my coding - however, this only allows the initiation of the mutation counter. I am still unable to add the extra dataframes. Please see the editted code.
Theo Jones
CC BY-SA 4.0
2020-10-31T07:58:58.803Z
Would you be able to share two or three of your input files?
Bastian Schiffthaler
CC BY-SA 4.0
2020-10-31T12:12:46.323Z
https://docs.google.com/document/d/1fV6wiYKkTngCFOeOwcfl1JGbhpY_VpWrSpGsdTfy-bw/edit?usp=sharing
Lineage A

https://docs.google.com/document/d/1qCNtzRI6EV2GNn3BPuP6EtuB5VWkFqgmGuLT4z_XqoY/edit?usp=sharing
A222V Mutation
Theo Jones
CC BY-SA 4.0
2020-10-31T16:48:05.673Z
Sorry @BastianSchiffthaler I realise those two files may seem lacking. I was in a rush for work while sending these. So the ‘Dates’ and ‘Location’ input files are identical to the ‘Lineage’ files. I merely copied these, as the code didn’t like trying to make multiple separations from the same folder for some reason
Theo Jones
<covid-19><sars-cov-2><python><pandas><dataframe>
Adding multiple dataframes to compare against genotypes - DATE/LOCATION - geotemporal incidences of SARS2 genotypes
# Adding multiple dataframes to compare against genotypes - DATE/LOCATION - geotemporal incidences of SARS2 genotypes

(Asked by: username_0 on 2020-10-29)

I'm trying to reuse some code I have previously used to add extra parameters.
In the initial code, it was used to give a 1/0 counter if a specific mutation was present (first code below)
import re
import os
import pandas as pd
import logging
import sys
import csv

root = "C:"

os.chdir(root)

lineages = os.listdir('Lineages')
mutations = os.listdir('Mutations')

for lineage in lineages:
    
    df = pd.read_csv('Lineages/' + lineage).dropna()
    df.columns = ["lineage"]
    df['lineage'] = [x.split('|')[0].split('_')[-1]
                     for x in df['lineage']]
               

    for mutation in mutations:
        
        mutation_df = pd.read_csv('Mutations/' + mutation).dropna()
        mutation_header = mutation.replace(".txt", "")
        mutation_df.columns = [mutation_header]
        mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] 
                          for x in mutation_df[mutation_header]]
        
        mutation_set = set(mutation_df[mutation_header])
        
        df[mutation_header] = [1 if x in mutation_set else 0
                               for x in df['lineage']]
        
    df.to_csv('Results9/' + lineage.replace(".txt", "") + '.csv',
                 header = True, 
                 index = False)

However, I now wish to add extra parameters such as date of sample and location of sample.
The first code produces this type of counter system:
lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A
417941,0,1,0,0,0,0,0,0,0,0,0
417942,0,1,0,0,0,0,0,0,0,0,0
417944,0,1,0,0,0,0,0,0,0,0,0
417946,0,1,0,0,0,0,0,0,0,0,0
417947,0,1,0,0,0,0,0,0,0,0,0
417948,0,1,0,0,0,0,0,0,0,0,0
417950,0,0,0,0,0,0,0,0,0,0,0
417955,0,1,0,0,0,0,0,0,0,0,0
431011,0,1,0,0,0,0,0,0,0,0,0

This code is fine and working, but if I try and add the second parameter (date/location) with the following code:
import re
import os
import pandas as pd
import logging
import sys
import csv

root = "C:/Users/jones/Downloads/SARSCOV2/"

os.chdir(root)

lineages = os.listdir('Lineages')
mutations = os.listdir('Mutations')
dates = os.listdir('Dates')
locations = os.listdir('Locations')

for date in dates:
            
    date_df = pd.read_csv('Dates/' + date).dropna()
    date_df.columns = ["Date"]
    date_df['Date'] = [x.split('|')[-1].split('/')[-1] 
                       for x in date_df['Date']]
            
    for location in locations:
                
        location_df = pd.read_csv('Locations/' + location).dropna()
        location_df.columns = ["Location"]
        location_df['Location'] = [x.split('/')[1].split('_')[0]
                                   for x in location_df['Location']]
            
        for lineage in lineages:
    
            df = pd.read_csv('Lineages/' + lineage).dropna()
            df.columns = ["lineage"]
            df['lineage'] = [x.split('_')[-1].split('|')[0]
                             for x in df['lineage']]
               

            for mutation in mutations:
        
                mutation_df = pd.read_csv('Mutations/' + mutation).dropna()
                mutation_header = mutation.replace(".txt", "")
                mutation_df.columns = [mutation_header]
                mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] 
                                                for x in mutation_df[mutation_header]]
        
                mutation_set = set(mutation_df[mutation_header])
        
                df[mutation_header] = [1 if x in mutation_set else 0
                                       for x in df['lineage']]
   
  
            df.to_csv('Results_3/' + lineage.replace(".txt", "") + '.csv',
                 header = True, 
                 index = False)

I retrieve this data, where the counting is maintaibned, but the new parameters (DATE/LOCATION) are not added
lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A
417941,0,1,0,0,0,0,0,0,0,0,0
417942,0,1,0,0,0,0,0,0,0,0,0
417944,0,1,0,0,0,0,0,0,0,0,0
417946,0,1,0,0,0,0,0,0,0,0,0
417947,0,1,0,0,0,0,0,0,0,0,0
417948,0,1,0,0,0,0,0,0,0,0,0
417950,0,0,0,0,0,0,0,0,0,0,0
417955,0,1,0,0,0,0,0,0,0,0,0
431011,0,1,0,0,0,0,0,0,0,0,0

In short, my end goal is to:

Add Lineage ID|Location of Sample|DD/MM/YYY|Mutations| -- as the headers
still maintain the mutation counter
Use the mutation counter as a guide to determine where and when the genotypes (mutation combinations) arose.

The files where I am retrieving the parameters looks like this also:

hCoV-19/Singapore/4/2020|EPI_ISL_410535|2020-02-03

hCoV-19/USA/WA13-UW9/2020|EPI_ISL_413601|2020-03-02

hCoV-19/USA/WA-UW142/2020|EPI_ISL_416680|2020-03-11

hCoV-19/USA/WA-UW143/2020|EPI_ISL_416681|2020-03-11

hCoV-19/USA/WA-UW144/2020|EPI_ISL_416682|2020-03-09

hCoV-19/USA/WA-UW145/2020|EPI_ISL_416683|2020-03-15

hCoV-19/USA/WA-UW146/2020|EPI_ISL_416684|2020-03-14

hCoV-19/USA/WA-UW147/2020|EPI_ISL_416685|2020-03-15

hCoV-19/USA/WA-UW148/2020|EPI_ISL_416686|2020-03-14

A very laborious, and poorly explained question; I apologise in advance!

**Tags:** covid-19, sars-cov-2, python, pandas, dataframe
> **Comments:**
> **username_0**: I have realised an error in my coding - however, this only allows the initiation of the mutation counter. I am still unable to add the extra dataframes. Please see the editted code.  
> **username_1**: Would you be able to share two or three of your input files?  
> **username_0**: https://docs.google.com/document/d/1fV6wiYKkTngCFOeOwcfl1JGbhpY_VpWrSpGsdTfy-bw/edit?usp=sharing Lineage A  https://docs.google.com/document/d/1qCNtzRI6EV2GNn3BPuP6EtuB5VWkFqgmGuLT4z_XqoY/edit?usp=sharing A222V Mutation  
> **username_0**: Sorry @BastianSchiffthaler I realise those two files may seem lacking. I was in a rush for work while sending these. So the ‘Dates’ and ‘Location’ input files are identical to the ‘Lineage’ files. I merely copied these, as the code didn’t like trying to make multiple separations from the same folder for some reason  

---

## 1 answers

### Answer by username_1 on 2020-10-31. Score: 2 | Accepted answer

Just to clarify, when I asked you to share some of your files, I meant to share them in a way that I can reproduce your issue from start to finish. You read from at least four different files in a quadruple for-loop and I'm pretty sure your goal could be achieved much more straightforwardly than your current implementation.
Without a reproducible example, I can offer a best-guess on what's happening: you create date_df and location_df, but after you instantiate them, you never do anything with the objects. Your final write happens from df, so you'll need to add those columns, e.g.:
df["Date"] = date_df["Date"][0]
df["Location"] = location_df["Location"][0]

The above is just an example, you'll need to make sure the correct value gets assigned to your df object instead of simply the first one.

> **Comments:**
> **username_0**: Again, apologies I was late running for work and rather skimmed the post. So would this be implemented before the mutation_set loop or between it? Or before the df[mutation_header]? I assume the latter  
> **username_0**: I assumed the latter and it’s worked perfectly? Thanks again!  

---

