<p>You will probably be interested in the following UCSC wiki page, which explains how to go from most of the UCSC tables to GTF/GFF:<br>
<a href="http://genomewiki.ucsc.edu/index.php/Genes_in_gtf_or_gff_format" rel="noreferrer">http://genomewiki.ucsc.edu/index.php/Genes_in_gtf_or_gff_format</a></p>

<p>The basic gist is that UCSC doesn't store any data internally as GTF or GFF, and so you will need to use our genePredToGtf utility in order to convert from our internally stored genePred format into GTF/GFF3.</p>

<p>This is the command I usually give when we get this question on our mailing list:</p>

<pre>
$ mysql --user=genome --host=genome-mysql.cse.ucsc.edu -A -N -e "select * from refGene" hg19 | cut -f2- | genePredToGtf -source=hg19.refGene.ucsc file stdin stdout
</pre>

<p>Change stdout to the output filename you want in the last command to get an hg19 refGene GTF file:</p>

<pre>
chr1    hg19.refGene.ucsc   transcript  11869   14362   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357";  gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    11869   12227   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "1"; exon_id "NR_148357.1"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    12613   12721   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "2"; exon_id "NR_148357.2"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    13221   14362   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "3"; exon_id "NR_148357.3"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   transcript  11874   14409   .   +   .   gene_id "DDX11L1"; transcript_id "NR_046018";  gene_name "DDX11L1";
...
</pre>

<p>If you have further questions about UCSC data or tools feel free to send your question to one of the below mailing lists:</p>

<ul>
<li>General questions: genome@soe.ucsc.edu</li>
<li>Questions involving private data: genome-www@soe.ucsc.edu</li>
<li>Questions involving mirror sites: genome-mirror@ose.ucsc.edu</li>
</ul>

CC BY-SA 3.0
2017-09-21T18:35:17.683Z
It would be rather nice if the "GTF" format output from the UCSC table browser simply returned exactly this (including the `gene` entries, which are always missing). This is the sort of thing people are actually expecting from it.
Devon Ryan
CC BY-SA 3.0
2017-09-21T19:14:01.417Z
I agree that this would be ideal. The problem is that the gene_id's returned may not be what people want. For example, if you were trying to get a GTF output for the wgEncodeGencodeBasicV27 table, would you want a gene_id like DDX11L1 or a gene_id like ENSG*? The other issue is the plain genePred table format doesn't even have a name2 field, so what do we assign as gene_id in this case? What if the table is some other variation of genePred that also doesn't have a name2? We use many different table variations, and it is hard to support a GTF output for all of them.
Christopher Lee
CC BY-SA 3.0
2017-09-22T16:12:48.097Z
@ChristopherLee, that's perfect, thank you. Regarding your response to @DevonRyan's comment, I can appreciate that it's difficult (and maybe unrealistic) to support all possible variations, but I'd also argue that HUGO is probably what most people want. In either case, I'm certain that reporting the transcript ID for *both* `gene_id` and `transcript_id` is not ideal! :-)
Mark Ebbert
CC BY-SA 3.0
2017-09-21T18:11:33.103Z
2017-09-21T18:11:33.103Z
Christopher Lee
<p>I'm not sure what I'm missing, but I'm struggling to find an official <code>hg38</code> <code>GTF</code> file with RefSeq annotations. I'd like to provide the <code>GTF</code> to <a href="https://combine-lab.github.io/salmon/" rel="nofollow noreferrer">Salmon</a> to get gene-level annotations. </p>

<p>Here's Salmon's help info for <code>--geneMap</code>:</p>

<blockquote>
  <p>File containing a mapping of
                                          transcripts to genes.  If this file is
                                          provided Salmon will output both
                                          quant.sf and quant.genes.sf files,
                                          where the latter contains aggregated
                                          gene-level abundance estimates.  The
                                          transcript to gene mapping should be
                                          provided as either a GTF file, or a in
                                          a simple tab-delimited format where
                                          each line contains the name of a
                                          transcript and the gene to which it
                                          belongs separated by a tab.  The
                                          extension of the file is used to
                                          determine how the file should be
                                          parsed.  Files ending in '.gtf', '.gff'
                                          or '.gff3' are assumed to be in GTF
                                          format; files with any other extension
                                          are assumed to be in the simple format.
                                          In GTF / GFF format, the
                                          "transcript_id" is assumed to contain
                                          the transcript identifier and the
                                          "gene_id" is assumed to contain the
                                          corresponding gene identifier.</p>
</blockquote>

<p>Here's what I've tried:</p>

<ol>
<li>NCBI seems to only provide <code>GFF</code> files. I know you can convert between the two, but that hasn't worked well for me in the past. Both file formats allow a lot of freedom, which makes conversions sloppy. Salmon, specifically, is looking for the <code>gene_id</code> and <code>transcript_id</code> annotations.</li>
<li>UCSC provides <code>GTF</code> files from RefSeq, but the <code>gene_id</code> annotation is identical to the <code>transcript_id</code> annotation (i.e., it's the NM number). Or maybe there's an option I'm missing.</li>
<li>Ensembl provides exactly the format and all of the information I could hope for, except the <code>transcript_id</code> is the Ensembl ID (ENST#), not RefSeq (NM#).</li>
</ol>

<p>Can anyone tell me how to get a <code>GTF</code> file with hg38 RefSeq annotations?</p>

CC BY-SA 3.0
2017-09-21T07:17:31.500Z
refseq IDs are per-transcript, so it's expected that the gene and transcript IDs will be the same. Given that, what would you like as the gene_id?
Devon Ryan
CC BY-SA 3.0
2017-09-21T14:25:10.060Z
@Devon Ryan I'm guessing the EntrezGene ID or HUGO symbol?
Ian Sudbery
CC BY-SA 3.0
2017-09-21T14:29:50.780Z
@Mark Ebbert: What are your down stream applications? The as far as I'm aware, the salmon output doesn't include the gene_id, so it shouldn't matter that it is the same as the transcript_id.
Ian Sudbery
CC BY-SA 3.0
2017-09-21T15:19:55.237Z
I appreciate your help. I added Salmon's help on `--geneMap`. They expect `gene_id` to contain a "gene identifier". I understand that could be several things, but I would prefer the HUGO symbol, as @IanSudbery suggested. In my perusing, I thing Salmon should probably target the `gene_name`, at least how Ensembl formats the GTF. In either case, I'm surprised by @DevonRyan's comment that the `gene_id` would equal the `transcript_id`. Not arguing that's the case, but what good does that do anyone?? :-)
Mark Ebbert
CC BY-SA 3.0
2017-09-21T15:26:28.733Z
@IanSudbery, I ultimately want to aggregate transcript expression into the parent genes. I found last night that one of the Salmon authors now recommend `tximport` (https://github.com/COMBINE-lab/salmon/issues/98), which I will use. But this isn't the first time I've "needed" a RefSeq GTF and couldn't find one. Just surprises me, but maybe that indicates I need to assess my approach?
Mark Ebbert
<ngs><public-databases><sequence-annotation><human-genome><gtf>
hg38 GTF file with RefSeq annotations
# hg38 GTF file with RefSeq annotations

(Asked by: username_0 on 2017-09-21)

I'm not sure what I'm missing, but I'm struggling to find an official `hg38` `GTF` file with RefSeq annotations. I'd like to provide the `GTF` to [Salmon](https://combine-lab.github.io/salmon/) to get gene-level annotations.

Here's Salmon's help info for `--geneMap`:

> File containing a mapping of
> transcripts to genes. If this file is
> provided Salmon will output both
> quant.sf and quant.genes.sf files,
> where the latter contains aggregated
> gene-level abundance estimates. The
> transcript to gene mapping should be
> provided as either a GTF file, or a in
> a simple tab-delimited format where
> each line contains the name of a
> transcript and the gene to which it
> belongs separated by a tab. The
> extension of the file is used to
> determine how the file should be
> parsed. Files ending in '.gtf', '.gff'
> or '.gff3' are assumed to be in GTF
> format; files with any other extension
> are assumed to be in the simple format.
> In GTF / GFF format, the
> "transcript\_id" is assumed to contain
> the transcript identifier and the
> "gene\_id" is assumed to contain the
> corresponding gene identifier.

Here's what I've tried:

1. NCBI seems to only provide `GFF` files. I know you can convert between the two, but that hasn't worked well for me in the past. Both file formats allow a lot of freedom, which makes conversions sloppy. Salmon, specifically, is looking for the `gene_id` and `transcript_id` annotations.
2. UCSC provides `GTF` files from RefSeq, but the `gene_id` annotation is identical to the `transcript_id` annotation (i.e., it's the NM number). Or maybe there's an option I'm missing.
3. Ensembl provides exactly the format and all of the information I could hope for, except the `transcript_id` is the Ensembl ID (ENST#), not RefSeq (NM#).

Can anyone tell me how to get a `GTF` file with hg38 RefSeq annotations?

**Tags:** ngs, public-databases, sequence-annotation, human-genome, gtf
> **Comments:**
> **username_1**: refseq IDs are per-transcript, so it's expected that the gene and transcript IDs will be the same. Given that, what would you like as the gene_id?  
> **username_2**: @Devon Ryan I'm guessing the EntrezGene ID or HUGO symbol?  
> **username_2**: @Mark Ebbert: What are your down stream applications? The as far as I'm aware, the salmon output doesn't include the gene_id, so it shouldn't matter that it is the same as the transcript_id.  
> **username_0**: I appreciate your help. I added Salmon's help on `--geneMap`. They expect `gene_id` to contain a "gene identifier". I understand that could be several things, but I would prefer the HUGO symbol, as @IanSudbery suggested. In my perusing, I thing Salmon should probably target the `gene_name`, at least how Ensembl formats the GTF. In either case, I'm surprised by @DevonRyan's comment that the `gene_id` would equal the `transcript_id`. Not arguing that's the case, but what good does that do anyone?? :-)  
> **username_0**: @IanSudbery, I ultimately want to aggregate transcript expression into the parent genes. I found last night that one of the Salmon authors now recommend `tximport` (https://github.com/COMBINE-lab/salmon/issues/98), which I will use. But this isn't the first time I've "needed" a RefSeq GTF and couldn't find one. Just surprises me, but maybe that indicates I need to assess my approach?  

---

## 1 answers

### Answer by username_3 on 2017-09-21. Score: 5 | Accepted answer

You will probably be interested in the following UCSC wiki page, which explains how to go from most of the UCSC tables to GTF/GFF:
http://genomewiki.ucsc.edu/index.php/Genes_in_gtf_or_gff_format
The basic gist is that UCSC doesn't store any data internally as GTF or GFF, and so you will need to use our genePredToGtf utility in order to convert from our internally stored genePred format into GTF/GFF3.
This is the command I usually give when we get this question on our mailing list:

$ mysql --user=genome --host=genome-mysql.cse.ucsc.edu -A -N -e "select * from refGene" hg19 | cut -f2- | genePredToGtf -source=hg19.refGene.ucsc file stdin stdout

Change stdout to the output filename you want in the last command to get an hg19 refGene GTF file:

chr1    hg19.refGene.ucsc   transcript  11869   14362   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357";  gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    11869   12227   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "1"; exon_id "NR_148357.1"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    12613   12721   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "2"; exon_id "NR_148357.2"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   exon    13221   14362   .   +   .   gene_id "LOC102725121"; transcript_id "NR_148357"; exon_number "3"; exon_id "NR_148357.3"; gene_name "LOC102725121";
chr1    hg19.refGene.ucsc   transcript  11874   14409   .   +   .   gene_id "DDX11L1"; transcript_id "NR_046018";  gene_name "DDX11L1";
...

If you have further questions about UCSC data or tools feel free to send your question to one of the below mailing lists:

General questions: genome@soe.ucsc.edu
Questions involving private data: genome-www@soe.ucsc.edu
Questions involving mirror sites: genome-mirror@ose.ucsc.edu

> **Comments:**
> **username_1**: It would be rather nice if the "GTF" format output from the UCSC table browser simply returned exactly this (including the `gene` entries, which are always missing). This is the sort of thing people are actually expecting from it.  
> **username_3**: I agree that this would be ideal. The problem is that the gene_id's returned may not be what people want. For example, if you were trying to get a GTF output for the wgEncodeGencodeBasicV27 table, would you want a gene_id like DDX11L1 or a gene_id like ENSG*? The other issue is the plain genePred table format doesn't even have a name2 field, so what do we assign as gene_id in this case? What if the table is some other variation of genePred that also doesn't have a name2? We use many different table variations, and it is hard to support a GTF output for all of them.  
> **username_0**: @ChristopherLee, that's perfect, thank you. Regarding your response to @DevonRyan's comment, I can appreciate that it's difficult (and maybe unrealistic) to support all possible variations, but I'd also argue that HUGO is probably what most people want. In either case, I'm certain that reporting the transcript ID for *both* `gene_id` and `transcript_id` is not ideal! :-)  

---

