<p>If you just want the number of unique headers, you can do this:</p>

<pre><code>grep '&gt;' reference.fasta | sort | uniq | wc -l
</code></pre>

<p>If you want a list of the unique headers, you can do this:</p>

<pre><code>grep '&gt;' reference.fasta | sort | uniq
</code></pre>

<p>If you want a histogram of how many times each header occurs, you can do this:</p>

<pre><code>grep '&gt;' reference.fasta | sort | uniq -c | awk '{printf("%s\t%s\n", <span class="math-container">$1, $</span>2)}'
</code></pre>

CC BY-SA 4.0
2018-10-10T03:14:54.157Z
2018-10-10T08:24:08.857Z
2018-10-10T08:24:08.857Z
conchoecia
<p>The <code>uniq</code> command expects sorted input. Interestingly, the <code>sort</code> command actually has a "unique" option, <code>-u</code>, which means <code>uniq</code> is not strictly needed. For the fastest processing, you can look for the '>' character at the start of lines with grep:</p>

<pre><code>grep '^&gt;' reference.fasta | sort -u &gt; reference_headers_unique.fasta
</code></pre>

<p>For returning the <em>number</em> of unique lines, pipe through <code>wc -l</code>:</p>

<pre><code>grep '^&gt;' reference.fasta | sort -u | wc -l
</code></pre>

<p>For more information about regular expressions, see <a href="https://www.gnu.org/software/findutils/manual/html_node/find_html/grep-regular-expression-syntax.html#grep-regular-expression-syntax" rel="nofollow noreferrer">here</a>.</p>

CC BY-SA 4.0
2018-10-10T09:39:45.163Z
This seems like the best solution given, just note it assumes that the desired output order for the unique headers is the order provided by `sort` (which may not be stable depending on the locale)
Chris_Rands
CC BY-SA 4.0
2018-10-10T04:19:22.000Z
2018-10-10T04:19:22.000Z
gringer
<p>If, for some reason, you need to keep the original order of the input file, so you only want the <em>first</em> occurrence of each header, you can do:</p>

<pre><code>awk '/^&gt;/ &amp;&amp; !a[$0]++' reference.fasta &gt; headers
</code></pre>

<p>Or, to get the number only:</p>

<pre><code>awk '/^&gt;/ &amp;&amp; !a[$0]++{k++}; END{print k}' reference.fasta 
</code></pre>

<p>However, these only make sense if you need to keep the order of the original file. If not, use <a href="https://bioinformatics.stackexchange.com/a/5188/298">Gringer's <code>grep</code> approach</a> which will be much faster (<code>reference.fasta</code> is hg19):</p>

<pre><code><span class="math-container">$ time awk '/^&gt;/ &amp;&amp; !a[$</span>0]++{k++}; END{print k}' reference.fasta 
93

real    0m34.510s
user    0m31.861s
sys     0m2.634s

$ time grep '^&gt;' reference.fasta | sort -u | wc -l
93

real    0m16.597s
user    0m0.820s
sys     0m3.725s
</code></pre>

CC BY-SA 4.0
2018-10-12T07:53:07.073Z
+1 for order capturing approach and benchmark, I guess ripgrep might be even faster
Chris_Rands
CC BY-SA 4.0
2018-10-10T12:57:22.167Z
2018-10-11T07:58:07.463Z
2018-10-11T07:58:07.463Z
terdon
<p>I tried to use the following command</p>

<pre><code>uniq -u reference.fasta &gt;&gt; reference_uniq.fasta
</code></pre>

<p>I'd like a count of the unique headers.</p>

CC BY-SA 4.0
2018-10-10T08:39:15.590Z
does it matter what the output order of the unique headers is?
Chris_Rands
<fasta><linux>
finding unique headers in a fasta file using linux command line
# finding unique headers in a fasta file using linux command line

(Asked by: username_0 on 2018-10-10)

I tried to use the following command
uniq -u reference.fasta >> reference_uniq.fasta

I'd like a count of the unique headers.

**Tags:** fasta, linux
> **Comments:**
> **username_1**: does it matter what the output order of the unique headers is?  

---

## 3 answers

### Answer by username_2 on 2018-10-10. Score: 5

If you just want the number of unique headers, you can do this:
grep '>' reference.fasta | sort | uniq | wc -l

If you want a list of the unique headers, you can do this:
grep '>' reference.fasta | sort | uniq

If you want a histogram of how many times each header occurs, you can do this:
grep '>' reference.fasta | sort | uniq -c | awk '{printf("%s\t%s\n", $1, $2)}'


---

### Answer by username_3 on 2018-10-10. Score: 3

The uniq command expects sorted input. Interestingly, the sort command actually has a "unique" option, -u, which means uniq is not strictly needed. For the fastest processing, you can look for the '>' character at the start of lines with grep:
grep '^>' reference.fasta | sort -u > reference_headers_unique.fasta

For returning the number of unique lines, pipe through wc -l:
grep '^>' reference.fasta | sort -u | wc -l

For more information about regular expressions, see here.

> **Comments:**
> **username_1**: This seems like the best solution given, just note it assumes that the desired output order for the unique headers is the order provided by `sort` (which may not be stable depending on the locale)  

---

### Answer by username_4 on 2018-10-10. Score: 1

If, for some reason, you need to keep the original order of the input file, so you only want the first occurrence of each header, you can do:
awk '/^>/ && !a[$0]++' reference.fasta > headers

Or, to get the number only:
awk '/^>/ && !a[$0]++{k++}; END{print k}' reference.fasta 

However, these only make sense if you need to keep the order of the original file. If not, use Gringer's grep approach which will be much faster (reference.fasta is hg19):
$ time awk '/^>/ && !a[$0]++{k++}; END{print k}' reference.fasta 
93

real    0m34.510s
user    0m31.861s
sys     0m2.634s

$ time grep '^>' reference.fasta | sort -u | wc -l
93

real    0m16.597s
user    0m0.820s
sys     0m3.725s

> **Comments:**
> **username_1**: +1 for order capturing approach and benchmark, I guess ripgrep might be even faster  

---

