<p>Keep it simple:</p>

<pre><code>awk '!a[$1]++' file.txt 
</code></pre>

<p>If you have a different command line separator you need to specify with the <code>-F</code> option.</p>

<p>Space</p>

<pre><code>awk -F' ' '!a[$1]++' text.txt 
</code></pre>

<p>Tab</p>

<pre><code>awk -F'\t' '!a[$1]++' text.txt 
</code></pre>

<p>If you don't have the first column in order maybe should be a good idea sort first. </p>

<pre><code>sort text.txt | awk -F' ' '!a[$1]++'
</code></pre>

<p>The name of the array in this case is <code>a</code>. There is a better explanation in <a href="https://superuser.com/a/1107659">this post</a>.</p>

CC BY-SA 4.0
2020-02-18T18:02:08.557Z
2020-02-18T18:02:08.557Z
zorbax
<p>That sound like a job for a perl one-liner:</p>

<pre><code>perl -ane 'unless(exists <span class="math-container">$seen{$F[0]}){print $_; $seen{$</span>F[0]}++}'
</code></pre>

<p>Explanation:<br/> 
The input is read line-wise and printed back to the command line if the ID in the first column has not been seen before. </p>

<ul>
<li><code>perl -ane '</code> reads input line-wise and splits fields to array @F</li>
<li><code>unless(exists $seen{$F[0]}){</code> checks if ID exists in hash %seen and continues if that is not the case</li>
<li><code>print $_;</code> prints the original input to STDOUT </li>
<li><code>$seen{$F[0]}++}'</code> adds the ID to the hash %seen </li>
</ul>

CC BY-SA 4.0
2020-02-18T09:59:51.153Z
2020-02-18T09:59:51.153Z
Wayne_Yux
<p>I have a file with Ensembl gene and Transcript IDs :</p>

<blockquote>
  <p><a href="https://i.sstatic.net/Dg9J0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Dg9J0.png" alt="enter image description here"></a></p>
</blockquote>

<p>I want to retain only the first entires from both the column so that the output looks like this:</p>

<pre><code>ENSG00000003137 ENST00000001146
ENSG00000003402 ENST00000309955
</code></pre>

<p>and ignore the rest. 
Have tried awk <code>awk '!a[$1$2]++'</code> and versions of it. Nothing works.
Kindly suggest some way to get the desired output.</p>

<p>regards.</p>

<ensembl>
select unique names while removing the duplicates from a column
# select unique names while removing the duplicates from a column

(Asked by: username_0 on 2020-02-17)

I have a file with Ensembl gene and Transcript IDs :



I want to retain only the first entires from both the column so that the output looks like this:
ENSG00000003137 ENST00000001146
ENSG00000003402 ENST00000309955

and ignore the rest. 
Have tried awk awk '!a[$1$2]++' and versions of it. Nothing works.
Kindly suggest some way to get the desired output.
regards.

**Tags:** ensembl

---

## 2 answers

### Answer by username_1 on 2020-02-18. Score: 1 | Accepted answer

Keep it simple:
awk '!a[$1]++' file.txt 

If you have a different command line separator you need to specify with the -F option.
Space
awk -F' ' '!a[$1]++' text.txt 

Tab
awk -F'\t' '!a[$1]++' text.txt 

If you don't have the first column in order maybe should be a good idea sort first. 
sort text.txt | awk -F' ' '!a[$1]++'

The name of the array in this case is a. There is a better explanation in this post.


---

### Answer by username_2 on 2020-02-18. Score: 1

That sound like a job for a perl one-liner:
perl -ane 'unless(exists $seen{$F[0]}){print $_; $seen{$F[0]}++}'

Explanation: 
The input is read line-wise and printed back to the command line if the ID in the first column has not been seen before. 

perl -ane ' reads input line-wise and splits fields to array @F
unless(exists $seen{$F[0]}){ checks if ID exists in hash %seen and continues if that is not the case
print $_; prints the original input to STDOUT 
$seen{$F[0]}++}' adds the ID to the hash %seen


---

