<p>A member of the SRA submission staff pointed out that using</p>

<pre><code>prefetch --type all SRR5253957
</code></pre>

<p>will download the original files. In this case, it means running the above within an EC2 instance colocated with the S3 bucket (so, <strong>us-east-1</strong>) and having installed and configured SRA Toolkit to work from AWS (<a href="https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/" rel="nofollow noreferrer">as per this documentation</a>).</p>

<p>Unfortunately, the particular files I am concerned with are not currently accessible, but this should generically work in most situations.</p>

CC BY-SA 4.0
2020-04-17T22:52:52.590Z
2020-04-17T22:52:52.590Z
merv
<p>I want to download the original BAM files that the authors had uploaded to SRA. Normally, I would just use <code>sam-dump</code>, but the files are having issues that seem related to <a href="https://github.com/ncbi/sra-tools/issues/23" rel="nofollow noreferrer">this issue</a>. Since according to the entry, AWS S3 also hosts the original BAM files, I thought I could download these directly.</p>

<h2>Example SRA Entry</h2>

<p><img src="https://i.sstatic.net/LP8Y7.png" alt="enter image description here"></p>

<p>NCBI documentation implies that I can't download this directly, but I can freely copy to other AWS locations within the region. To this end, I created my own S3 bucket (<code>mm-mneuron</code>) and am now trying to copy from the SRA bucket to mine.  Here's what I try:</p>

<pre><code>import boto3
import botocore

s3 = boto3.resource('s3')

bam_file = {
  'Bucket': 'sra-pub-src-6',
  'Key': 'SRR5253957/RPI25_0.bam'
}

my_bucket = s3.Bucket('mm-mneuron')

my_bucket.copy(bam_file, 'RPI25_0.bam')
</code></pre>

<p>This fails with:</p>

<blockquote>
  <p>botocore.exceptions.ClientError: An error occurred (403) when calling
  the HeadObject operation: Forbidden</p>
</blockquote>

<p>That is, it sounds like I can't access the SRA bucket. I've tested downloading and uploading to my bucket, so I know I have write permissions. Not sure what else to try here.</p>

<p><strong>How can I access the SRA data on S3?</strong> </p>

CC BY-SA 4.0
2020-04-17T23:57:55.610Z
It seems like is not a public bucket. You should download inside an (SRA AMI)[https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/] from sra-pub-src-6 to your instance, and then, in you local terminal download from SRA AMI in EC2 to your computer.
zorbax
CC BY-SA 4.0
2020-04-18T04:21:34.847Z
@zorbax yes that's what I did end up doing ([see answer](https://bioinformatics.stackexchange.com/a/12962/900)), but there is something specifically wrong with the deposits I'm trying to download. In general, are SRA buckets only accessible through SRA tools? i.e., one has to run an EC2 instance? It seems like direct transfers across S3 buckets would an obvious interface to provide.
merv
CC BY-SA 4.0
2020-04-23T13:35:00.220Z
You only have access to the public data sets with s3, and the key prj_phs710EA_test.ngc in EC2 only bring acces to 1000 Genomes data. https://www.biostars.org/p/393172/
zorbax
<rna-seq><python><phylogenetics><aws>
Downloading SRA Files from AWS
# Downloading SRA Files from AWS

(Asked by: username_0 on 2020-04-16)

I want to download the original BAM files that the authors had uploaded to SRA. Normally, I would just use sam-dump, but the files are having issues that seem related to this issue. Since according to the entry, AWS S3 also hosts the original BAM files, I thought I could download these directly.
Example SRA Entry

NCBI documentation implies that I can't download this directly, but I can freely copy to other AWS locations within the region. To this end, I created my own S3 bucket (mm-mneuron) and am now trying to copy from the SRA bucket to mine.  Here's what I try:
import boto3
import botocore

s3 = boto3.resource('s3')

bam_file = {
  'Bucket': 'sra-pub-src-6',
  'Key': 'SRR5253957/RPI25_0.bam'
}

my_bucket = s3.Bucket('mm-mneuron')

my_bucket.copy(bam_file, 'RPI25_0.bam')

This fails with:

botocore.exceptions.ClientError: An error occurred (403) when calling
  the HeadObject operation: Forbidden

That is, it sounds like I can't access the SRA bucket. I've tested downloading and uploading to my bucket, so I know I have write permissions. Not sure what else to try here.
How can I access the SRA data on S3?

**Tags:** rna-seq, python, phylogenetics, aws
> **Comments:**
> **username_1**: It seems like is not a public bucket. You should download inside an (SRA AMI)[https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/] from sra-pub-src-6 to your instance, and then, in you local terminal download from SRA AMI in EC2 to your computer.  
> **username_0**: @zorbax yes that's what I did end up doing ([see answer](https://bioinformatics.stackexchange.com/a/12962/900)), but there is something specifically wrong with the deposits I'm trying to download. In general, are SRA buckets only accessible through SRA tools? i.e., one has to run an EC2 instance? It seems like direct transfers across S3 buckets would an obvious interface to provide.  
> **username_1**: You only have access to the public data sets with s3, and the key prj_phs710EA_test.ngc in EC2 only bring acces to 1000 Genomes data. https://www.biostars.org/p/393172/  

---

## 1 answers

### Answer by username_0 on 2020-04-17. Score: 2

A member of the SRA submission staff pointed out that using
prefetch --type all SRR5253957

will download the original files. In this case, it means running the above within an EC2 instance colocated with the S3 bucket (so, us-east-1) and having installed and configured SRA Toolkit to work from AWS (as per this documentation).
Unfortunately, the particular files I am concerned with are not currently accessible, but this should generically work in most situations.


---

