<p>I believe a <code>pandas</code> approach would be fast enough (probably faster than any other Python approach without pandas).</p>

<p>After converting your table to a <code>pandas data frame</code>, you can do something like <code>df.filter(regex=(".*Chr01"))</code> for the first chromosome then write the resulting data to file with the <code>to_csv()</code> method.</p>

<pre><code>import pandas as pd

data = pd.read_csv('input.txt', delimiter = "\t", index_col = False)

for i in ["Chr01", "Chr02", "Chr03"]:
    regex_pattern = ".*" + i
    print(data.filter(regex=regex_pattern))
</code></pre>

CC BY-SA 4.0
2019-10-10T07:28:02.960Z
2019-10-10T21:44:46.147Z
2019-10-10T21:44:46.147Z
haci
<p>I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me?
I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual). </p>

<p>The first row is the SNPs_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file.
I have done it with "R", but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?</p>

<p>here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)</p>

<pre><code>"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280"....... "Ha412HOChr17:303402"
0 1 0 0 ......0
0 1 0 0 ......0
0 1 0 0 ......0
0 2 0 0 ......0
0 1 1 1 ......1
0 2 0 0 ......0
</code></pre>

<p>and this is an example of my desired output for example for chromosome 1:</p>

<pre><code>out.chrom1
"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280" 
0 1 0 0 
0 1 0 0
0 1 0 0 
0 2 0 0 
0 1 1 1 
0 2 0 0 
0 0 0 0 
0 2 0 0
0 1 2 2 
</code></pre>

CC BY-SA 4.0
2019-10-10T04:32:32.587Z
Hi, good question but perhaps post your R code?
M__
CC BY-SA 4.0
2019-10-10T14:27:28.167Z
Are the leading spaces part of your input file? Does the second row, for example really start with 4 spaces? If not, please [edit] and remove them to avoid confusion. Also, are those really the IDs? Do they all start with `Ha412HO`, then `Chr` and number(s), followed by `:` and the position? Or can the `Ha412HO` change?
terdon
CC BY-SA 4.0
2019-10-10T15:26:47.920Z
I just edited my question according to your comment. And yes, the IDs start with Ha412HOChr followed by chromosome, the : and SNP position
Anna1364
<python><perl>
storing SNPs in the genotype file for each chromosome in a separate file
# storing SNPs in the genotype file for each chromosome in a separate file

(Asked by: username_0 on 2019-10-10)

I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me?
I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual). 
The first row is the SNPs_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file.
I have done it with "R", but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?
here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)
"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280"....... "Ha412HOChr17:303402"
0 1 0 0 ......0
0 1 0 0 ......0
0 1 0 0 ......0
0 2 0 0 ......0
0 1 1 1 ......1
0 2 0 0 ......0

and this is an example of my desired output for example for chromosome 1:
out.chrom1
"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280" 
0 1 0 0 
0 1 0 0
0 1 0 0 
0 2 0 0 
0 1 1 1 
0 2 0 0 
0 0 0 0 
0 2 0 0
0 1 2 2

**Tags:** python, perl
> **Comments:**
> **username_1**: Hi, good question but perhaps post your R code?  
> **username_2**: Are the leading spaces part of your input file? Does the second row, for example really start with 4 spaces? If not, please [edit] and remove them to avoid confusion. Also, are those really the IDs? Do they all start with `Ha412HO`, then `Chr` and number(s), followed by `:` and the position? Or can the `Ha412HO` change?  
> **username_0**: I just edited my question according to your comment. And yes, the IDs start with Ha412HOChr followed by chromosome, the : and SNP position  

---

## 1 answers

### Answer by username_3 on 2019-10-10. Score: 2

I believe a pandas approach would be fast enough (probably faster than any other Python approach without pandas).
After converting your table to a pandas data frame, you can do something like df.filter(regex=(".*Chr01")) for the first chromosome then write the resulting data to file with the to_csv() method.
import pandas as pd

data = pd.read_csv('input.txt', delimiter = "\t", index_col = False)

for i in ["Chr01", "Chr02", "Chr03"]:
    regex_pattern = ".*" + i
    print(data.filter(regex=regex_pattern))


---

