<p>I used a local version of <code>segmasker</code> <strong>*</strong>. One way to solve this is,</p>
<pre><code>perl -p -i -e 's/ //g' bioinfo.fa
perl -p -e 's/\t/^/g' bioinfo.fa | segmasker | perl -p -e 's/^/\t/g'  &gt; bioinfo2.sm
</code></pre>
<p><em>Edit: I changed <code>,</code> for <code>^</code> because the OPs code already has <code>,</code> and will insert an extra tab (\t). Same change in the line below</em></p>
<p>Here the input file you presented in the link is called <code>bioinfo.fa</code>. If you need the pipe explaining let me know, but it's
fairly straight forward.</p>
<p>If you want to preserve space characters, i.e. ' ' or &quot;\s&quot; - because in the code above I delete them - then (not tested):</p>
<pre><code>perl -p -e 's/\t/^/g' bioinfo.fa | perl -p -e 's/ /_/g' | segmasker | perl -p -e 's/_/ /g' | perl -p -e 's/^/\t/g'  &gt; bioinfo2.sm
</code></pre>
<p>The issue the OP encounter is not unique to tabs (\t) but also occurs with spaces (\s), at least with <code>makebasedb</code>.</p>
<p><strong>input</strong></p>
<pre><code>&gt;A0A000 IPR004839   Aminotransferase, class I/classII   PF00155 41  381
GVSAIDGMNTLVRGQKLPIFSSSGQPHSELAMQIARQASVPEEEEGGDDEEGSEFAVIFGAMGITQEEANEFMEDFERTGALERSVVFMNLADDPAVERTVTPRMVLTTAEYLAFEKDYHVLVILTDMTNYCEALREIGAAREEVPGRRGYPG
&gt;A0A000 IPR010961   Tetrapyrrole biosynthesis, 5-aminolevulinic acid synthase   TIGR01821   12  391
...
</code></pre>
<p><strong>output</strong></p>
<pre><code>&gt;A0A000 IPR004839       Aminotransferase        class_I/classII        PF00155 41      381
41 - 53
&gt;A0A000 IPR010961       Tetrapyrrole_biosynthesis       _5aminolevulinic_acid_synthase  TIGR01821       12      391
41 - 53
&gt;A0A000 IPR015421       Pyridoxal phosphate dependent transferase        major domainG3DSA:3.40.640.10  48      288
106 - 126
...
</code></pre>
<p>Please don't ask about solving this via an upstream <code>makeblastdb</code> because that gets complicated. The rate limiting step, i.e. why it is ignoring tabs is because of <code>makeblastdb</code>. Thats a consequence of the way its written ... its a developer issue (see <strong>blurb</strong>)</p>
<hr />
<p><strong>*</strong> Local <code>segmasker</code> installation ...</p>
<ol>
<li>Install <code>conda</code> via <code>Anaconda</code> or <code>miniconda</code></li>
<li><code>conda create -n blast3 python=3.11</code> (creating an environment ... yep 3.11 is out (wow!, if you don't understand why <strong>WOW!</strong> ask in chat)</li>
<li><code>conda activate blast3</code> (this is the conda environment, you can call it anything)</li>
<li><code>conda install -c bioconda blast</code></li>
</ol>
<p><code>perl | segmasker | perl</code> will now work. The Python installation is just good practice.</p>
<hr />
<p><strong>blurb about NCBI's blast</strong> The issue is <code>makeblastdb</code>, what happens is that long headers are a complete pain in algorithms because they use loads of RAM and you've got to carry them through reiterative loops (convergence etc ...). The way to solve it is via recoding (thats how I've done it). Essentially, you have a mini database forcing header recoding which activates at the start of the  algorithm and reassembles the original header at the end of the algorithm. The user never sees this step, but it's really important for efficiency. Thus within the reiteration the header isn't consuming RAM, because it's an internal code, beyond base 10 to squash the overhead even further. The thing is header can become a lot bigger than the sequence data and thats not cool when you've loads and loads of data, e.g. a blast database. NCBI just haven't bothered, like if you can't see it why do it attitude. What they've said is that 50 characters is the limitation of the header for <code>makeblastdb</code>... what NCBI is looking for is a unique id at the start of the header it then truncates everything beyond that, i.e. tabs, spaces ... anything it can grab. NCBI have carried this through into <code>segmasker</code> it's not needed, but <code>makeblastdb</code> and <code>segmasker</code> would normally be used together - thats why. In the OPs case this results in duplicate headers, so the file format falls over. This isn't the OPs fault.</p>

CC BY-SA 4.0
2022-11-28T20:46:53.587Z
Thank you for the exhaustive answer!
maciejwww
CC BY-SA 4.0
2022-11-28T20:48:41.380Z
I agree, **Python** 3.11 has many great features. What's your biggest WOW there?
maciejwww
CC BY-SA 4.0
2022-11-29T01:06:46.380Z
@maciejwww The criticism of Python is its slow. **3.11** is the major upgrade to focus on speed - its the fight back and may there be many more. Answer, you are welcome, note was a slight bug, I switched `,` for `^`.
M__
CC BY-SA 4.0
2022-11-28T09:46:17.040Z
2022-11-29T01:10:10.040Z
2022-11-29T01:10:10.040Z
M__
<p>I try to pass a <em>.fasta</em> file with headers where tabs are used as separators as input file to <a href="https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;changeset_revision=5edc472ec434" rel="nofollow noreferrer"><em>NCBI BLAST+ segmasker</em></a>:<br />
<code>&gt;A0A059VAR9    IPR022879   V-type ATP synthase regulatory subunit B/beta</code></p>
<p>Unfortunately, the <em>segmasker</em> output's headers are shortened to the first tab:<br />
<code>&gt;A0A059VAR9</code></p>
<p>Is there a way to get whole headers in the output file (without replacing all the tabs in the input file with white spaces)?</p>

CC BY-SA 4.0
2022-11-26T12:17:19.633Z
Could you supply a small data set please? You can fill is with dumbie amino acids ... I'll look at NCBI's`segmasker` local version because thats what I've got installed.
M__
CC BY-SA 4.0
2022-11-26T12:26:44.487Z
I think I also use local version. Here's a sample data: [link](https://pastebin.pl/view/ed83cade)
maciejwww
CC BY-SA 4.0
2022-11-28T09:50:00.577Z
Thanks for the file, answered below. Sorry about the delay. This question isn't simplistic.
M__
<fasta><phylogenetics><segmasker>
How to get full headers with tabs as separators with segmasker?
# How to get full headers with tabs as separators with segmasker?

(Asked by: username_0 on 2022-10-24)

I try to pass a *.fasta* file with headers where tabs are used as separators as input file to [*NCBI BLAST+ segmasker*](https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&changeset_revision=5edc472ec434):  
`>A0A059VAR9 IPR022879 V-type ATP synthase regulatory subunit B/beta`

Unfortunately, the *segmasker* output's headers are shortened to the first tab:  
`>A0A059VAR9`

Is there a way to get whole headers in the output file (without replacing all the tabs in the input file with white spaces)?

**Tags:** fasta, phylogenetics, segmasker
> **Comments:**
> **username_1**: Could you supply a small data set please? You can fill is with dumbie amino acids ... I'll look at NCBI's`segmasker` local version because thats what I've got installed.  
> **username_0**: I think I also use local version. Here's a sample data: [link](https://pastebin.pl/view/ed83cade)  
> **username_1**: Thanks for the file, answered below. Sorry about the delay. This question isn't simplistic.  

---

## 1 answers

### Answer by username_1 on 2022-11-28. Score: 1 | Accepted answer

I used a local version of segmasker *. One way to solve this is,
perl -p -i -e 's/ //g' bioinfo.fa
perl -p -e 's/\t/^/g' bioinfo.fa | segmasker | perl -p -e 's/^/\t/g'  > bioinfo2.sm

Edit: I changed , for ^ because the OPs code already has , and will insert an extra tab (\t). Same change in the line below
Here the input file you presented in the link is called bioinfo.fa. If you need the pipe explaining let me know, but it's
fairly straight forward.
If you want to preserve space characters, i.e. ' ' or "\s" - because in the code above I delete them - then (not tested):
perl -p -e 's/\t/^/g' bioinfo.fa | perl -p -e 's/ /_/g' | segmasker | perl -p -e 's/_/ /g' | perl -p -e 's/^/\t/g'  > bioinfo2.sm

The issue the OP encounter is not unique to tabs (\t) but also occurs with spaces (\s), at least with makebasedb.
input
>A0A000 IPR004839   Aminotransferase, class I/classII   PF00155 41  381
GVSAIDGMNTLVRGQKLPIFSSSGQPHSELAMQIARQASVPEEEEGGDDEEGSEFAVIFGAMGITQEEANEFMEDFERTGALERSVVFMNLADDPAVERTVTPRMVLTTAEYLAFEKDYHVLVILTDMTNYCEALREIGAAREEVPGRRGYPG
>A0A000 IPR010961   Tetrapyrrole biosynthesis, 5-aminolevulinic acid synthase   TIGR01821   12  391
...

output
>A0A000 IPR004839       Aminotransferase        class_I/classII        PF00155 41      381
41 - 53
>A0A000 IPR010961       Tetrapyrrole_biosynthesis       _5aminolevulinic_acid_synthase  TIGR01821       12      391
41 - 53
>A0A000 IPR015421       Pyridoxal phosphate dependent transferase        major domainG3DSA:3.40.640.10  48      288
106 - 126
...

Please don't ask about solving this via an upstream makeblastdb because that gets complicated. The rate limiting step, i.e. why it is ignoring tabs is because of makeblastdb. Thats a consequence of the way its written ... its a developer issue (see blurb)

* Local segmasker installation ...

Install conda via Anaconda or miniconda
conda create -n blast3 python=3.11 (creating an environment ... yep 3.11 is out (wow!, if you don't understand why WOW! ask in chat)
conda activate blast3 (this is the conda environment, you can call it anything)
conda install -c bioconda blast

perl | segmasker | perl will now work. The Python installation is just good practice.

blurb about NCBI's blast The issue is makeblastdb, what happens is that long headers are a complete pain in algorithms because they use loads of RAM and you've got to carry them through reiterative loops (convergence etc ...). The way to solve it is via recoding (thats how I've done it). Essentially, you have a mini database forcing header recoding which activates at the start of the  algorithm and reassembles the original header at the end of the algorithm. The user never sees this step, but it's really important for efficiency. Thus within the reiteration the header isn't consuming RAM, because it's an internal code, beyond base 10 to squash the overhead even further. The thing is header can become a lot bigger than the sequence data and thats not cool when you've loads and loads of data, e.g. a blast database. NCBI just haven't bothered, like if you can't see it why do it attitude. What they've said is that 50 characters is the limitation of the header for makeblastdb... what NCBI is looking for is a unique id at the start of the header it then truncates everything beyond that, i.e. tabs, spaces ... anything it can grab. NCBI have carried this through into segmasker it's not needed, but makeblastdb and segmasker would normally be used together - thats why. In the OPs case this results in duplicate headers, so the file format falls over. This isn't the OPs fault.

> **Comments:**
> **username_0**: Thank you for the exhaustive answer!  
> **username_0**: I agree, **Python** 3.11 has many great features. What's your biggest WOW there?  
> **username_1**: @maciejwww The criticism of Python is its slow. **3.11** is the major upgrade to focus on speed - its the fight back and may there be many more. Answer, you are welcome, note was a slight bug, I switched `,` for `^`.  

---

