<p>What you are doing makes <strong>no</strong> sense at all. A bigwig file contains intervals and then indicates the coverage of that interval. That coverage again comes from the alignments covering that region. There is no direct connection between the interval and the fastq file other than that the fastq file is the basis for the coverage calculation. Converting the interval to a fasta and then forcing it into a fastq file is technically possible, but has absolutely no practical meaning. You cannot recover sequencing reads from a bigwig, that information is lost once you convert your alignments (usually a BAM file) into a bigwig. You should open a new question describing the problem that this is actually based on.</p>

CC BY-SA 4.0
2022-06-26T19:37:49.680Z
I'll add slightly to this excellent answer and mention that if this is public data (highly likely), there is almost certainly a link somewhere to download the underlying fastq files.
Devon Ryan
CC BY-SA 4.0
2022-07-05T21:27:00.477Z
Excellent answer that misuses the word "convert" like most people on these forums do. One does not convert BAM to (big)wig, one extracts/computes wig information from a BAM file. "Convert" kind of implies information content preservation (convert sam to bam, etc.), which as you say in your answer is not true when getting WIG from BAM.
Ram RS
CC BY-SA 4.0
2022-06-26T13:59:37.797Z
2022-06-26T13:59:37.797Z
Siruis Blackhorse
<p>The code is verbose and I ain't at all sure some of it will work, at least in two places.</p>
<p>The first point is <code>os.path.expanduser</code> ... admittedly I prefer <code>pathlib</code>, however that is definitely not how I understand that command. My understanding of <code>expanduser</code> is if <code>~</code> is being used. You definitely want to look at <code>globb</code>-ing for the opening part of the code.</p>
<pre><code>from pathlib import Path
import os

mypath = Path(os.path.expanduser('~/Documents/H3K27ac'))
files = mypath.glob('*.bw') # or use ('*' + '.bw') 
</code></pre>
<p>I'm pretty certain that is what you want and its a lot easier than lines 1 to 10 in your code. The <code>files</code> list will then go straight into the loop.</p>
<p>Also, please check</p>
<pre><code>files = [x[:-3] if x.endswith('.bw') else x for x in files]
</code></pre>
<p>Comprehensions are never logical in Python but convention is always to place the <code>if</code> statement after the list of interest.</p>
<p>Thus the following is much closer:</p>
<pre><code>files = [x for x in files if not x.endswith('.bw') else x[:-3]]
</code></pre>
<p><strong>Note</strong> not tested, so this may throw a bug, but its better. However, using <code>glob</code> bing you don't need this statement.</p>
<p>The other issue is the sub.process call ... Normally they are written:</p>
<pre><code>os.chdir(path_bw)
file1 = 'file1.bw' 
file2 = 'fileout.wig'
subprocess.call([&quot;bigWigToWig&quot;,  file1, file2], shell=True)
</code></pre>
<p>The way you've written is:</p>
<ul>
<li>Really hard to read</li>
<li>I don't think it would work because of the absence of the <code>[]</code>, certainly if you are using <code>popen</code> it will not work</li>
</ul>
<p>Also check the following variant:</p>
<pre><code>subprocess.call([&quot;bigWigToWig&quot;,  file1, file2], text=True)
</code></pre>
<p>I know its seems a bit weird, but it very likely work (don't ask why)</p>
<p>I agree <code>subprocess.call</code> is a good idea in a Python pipeline. Issuing a hard path within <code>subprocess call</code> I'm not a fan hence using <code>os</code> to get Python to move into the <code>cwd</code> is a better approach.</p>
<p>A further point is the way you've used 'file' throughout your pipeline line, hmmmm... okay, but its difficult to read. Its not generally how a pipeline is written, but just saying.</p>
<p><code>nextflow</code> is an increasingly popular pipeline language and you may consider learning that rather than going down a Python route (Python is very good for data science), of course this particularly question could be very easily solved in <code>bash</code> (but for some reason it ain't trendy).</p>

CC BY-SA 4.0
2022-06-26T16:06:47.543Z
2022-06-26T16:35:12.513Z
2022-06-26T16:35:12.513Z
M__
<p>I am able to convert .bw files to .fq(fastq) manually in Shell, but I would like to automate the process coz I have hundreds of .bw file that I need to convert. Till now, I could think of following script, but unable to solve the problem. The .bw files are in H3K27ac folder.</p>
<pre><code>import subprocess

path_bw = &quot;~/Documents/H3K27ac&quot;
import os

files = os.path.expanduser(path_bw)
for file in files:
    if file.endswith('.gz'):
        files.remove(file) 
files = [x[:-3] if x.endswith('.bw') else x for x in files]


# for .bw files

for file in files:
    bigwigtowig = &quot;bigWigToWig &quot; + path_bw + file + &quot;.bw &quot; + path_bw + file +&quot;.wig&quot;
    subprocess.call(bigwigtowig, shell=True)

    wigtobed = &quot;wig2bed &lt; &quot; + path_bw + file + &quot;.wig &gt; &quot; + path_bw + file + &quot;.bed&quot;
    subprocess.call(wigtobed, shell=True)

    getfasta = &quot;bedtools getfasta -fi &quot; + path_bw + &quot;hg19.fa -bed &quot; + path_bw + file + &quot;.bed -fo &quot; + path_bw + file + &quot;.fa&quot;
    subprocess.call(getfasta, shell=True)

    perl = &quot;perl &quot; + path_bw + &quot;fasta_to_fastq.pl&quot; + path_bw + file + &quot;.fa &gt; &quot; + path_bw + file + &quot;.fq&quot;
    #os.system(perl)
    subprocess.run(perl, shell=True)
</code></pre>
<p>Manually, I used the following to convert the .bw files:</p>
<ol>
<li><code># bigWigToWig file1.bw fileout.wig</code></li>
<li><code># wig2bed &lt;fileout.wig&gt; fileout.bed</code></li>
<li><code># bedtools getfasta -fi hg19.fa -bed fileout.bed -fo fileout.fa</code></li>
<li><code># perl fasta_to_fastq.pl fileout.fa &gt; file1.fq</code></li>
</ol>
<p>All the above packages were preinstalled via shell and works perfectly. hg19 is the genom that i am using. fasta_to_fastq.pl is openly available in github and helps to convert .fa to .fq.</p>
<p>MAin thing i require is to convert all the files from .bw to .fq and this is the only way in my research that i could find. I need all the names to be the same as output.  For example file1.bw shoud be file1.fq , file2.bw --&gt; file2.fq and so on.</p>
<p>Do you have any ideas if this is possible or are there easier ways to convert it? It would be awesome to know your view point.</p>

CC BY-SA 4.0
2022-06-27T06:18:41.850Z
Can you please explain why you want to do this? It's a very unusual situation, and it's likely that there are better ways to approach what you want to do. It is often better to ask questions about your actual goal, rather than intermediate steps (see [here](https://en.wikipedia.org/wiki/XY_problem) for more information).
gringer
<python><fasta><gene><fastq><hg19>
Converting .bw files to .fq (fastq)
# Converting .bw files to .fq (fastq)

(Asked by: username_0 on 2022-06-26)

I am able to convert .bw files to .fq(fastq) manually in Shell, but I would like to automate the process coz I have hundreds of .bw file that I need to convert. Till now, I could think of following script, but unable to solve the problem. The .bw files are in H3K27ac folder.
import subprocess

path_bw = "~/Documents/H3K27ac"
import os

files = os.path.expanduser(path_bw)
for file in files:
    if file.endswith('.gz'):
        files.remove(file) 
files = [x[:-3] if x.endswith('.bw') else x for x in files]


# for .bw files

for file in files:
    bigwigtowig = "bigWigToWig " + path_bw + file + ".bw " + path_bw + file +".wig"
    subprocess.call(bigwigtowig, shell=True)

    wigtobed = "wig2bed < " + path_bw + file + ".wig > " + path_bw + file + ".bed"
    subprocess.call(wigtobed, shell=True)

    getfasta = "bedtools getfasta -fi " + path_bw + "hg19.fa -bed " + path_bw + file + ".bed -fo " + path_bw + file + ".fa"
    subprocess.call(getfasta, shell=True)

    perl = "perl " + path_bw + "fasta_to_fastq.pl" + path_bw + file + ".fa > " + path_bw + file + ".fq"
    #os.system(perl)
    subprocess.run(perl, shell=True)

Manually, I used the following to convert the .bw files:

# bigWigToWig file1.bw fileout.wig
# wig2bed  fileout.bed
# bedtools getfasta -fi hg19.fa -bed fileout.bed -fo fileout.fa
# perl fasta_to_fastq.pl fileout.fa > file1.fq

All the above packages were preinstalled via shell and works perfectly. hg19 is the genom that i am using. fasta_to_fastq.pl is openly available in github and helps to convert .fa to .fq.
MAin thing i require is to convert all the files from .bw to .fq and this is the only way in my research that i could find. I need all the names to be the same as output.  For example file1.bw shoud be file1.fq , file2.bw --> file2.fq and so on.
Do you have any ideas if this is possible or are there easier ways to convert it? It would be awesome to know your view point.

**Tags:** python, fasta, gene, fastq, hg19
> **Comments:**
> **username_1**: Can you please explain why you want to do this? It's a very unusual situation, and it's likely that there are better ways to approach what you want to do. It is often better to ask questions about your actual goal, rather than intermediate steps (see [here](https://en.wikipedia.org/wiki/XY_problem) for more information).  

---

## 2 answers

### Answer by username_2 on 2022-06-26. Score: 4

What you are doing makes **no** sense at all. A bigwig file contains intervals and then indicates the coverage of that interval. That coverage again comes from the alignments covering that region. There is no direct connection between the interval and the fastq file other than that the fastq file is the basis for the coverage calculation. Converting the interval to a fasta and then forcing it into a fastq file is technically possible, but has absolutely no practical meaning. You cannot recover sequencing reads from a bigwig, that information is lost once you convert your alignments (usually a BAM file) into a bigwig. You should open a new question describing the problem that this is actually based on.

> **Comments:**
> **username_3**: I'll add slightly to this excellent answer and mention that if this is public data (highly likely), there is almost certainly a link somewhere to download the underlying fastq files.  
> **username_4**: Excellent answer that misuses the word "convert" like most people on these forums do. One does not convert BAM to (big)wig, one extracts/computes wig information from a BAM file. "Convert" kind of implies information content preservation (convert sam to bam, etc.), which as you say in your answer is not true when getting WIG from BAM.  

---

### Answer by username_5 on 2022-06-26. Score: 1

The code is verbose and I ain't at all sure some of it will work, at least in two places.
The first point is os.path.expanduser ... admittedly I prefer pathlib, however that is definitely not how I understand that command. My understanding of expanduser is if ~ is being used. You definitely want to look at globb-ing for the opening part of the code.
from pathlib import Path
import os

mypath = Path(os.path.expanduser('~/Documents/H3K27ac'))
files = mypath.glob('*.bw') # or use ('*' + '.bw') 

I'm pretty certain that is what you want and its a lot easier than lines 1 to 10 in your code. The files list will then go straight into the loop.
Also, please check
files = [x[:-3] if x.endswith('.bw') else x for x in files]

Comprehensions are never logical in Python but convention is always to place the if statement after the list of interest.
Thus the following is much closer:
files = [x for x in files if not x.endswith('.bw') else x[:-3]]

Note not tested, so this may throw a bug, but its better. However, using glob bing you don't need this statement.
The other issue is the sub.process call ... Normally they are written:
os.chdir(path_bw)
file1 = 'file1.bw' 
file2 = 'fileout.wig'
subprocess.call(["bigWigToWig",  file1, file2], shell=True)

The way you've written is:

Really hard to read
I don't think it would work because of the absence of the [], certainly if you are using popen it will not work

Also check the following variant:
subprocess.call(["bigWigToWig",  file1, file2], text=True)

I know its seems a bit weird, but it very likely work (don't ask why)
I agree subprocess.call is a good idea in a Python pipeline. Issuing a hard path within subprocess call I'm not a fan hence using os to get Python to move into the cwd is a better approach.
A further point is the way you've used 'file' throughout your pipeline line, hmmmm... okay, but its difficult to read. Its not generally how a pipeline is written, but just saying.
nextflow is an increasingly popular pipeline language and you may consider learning that rather than going down a Python route (Python is very good for data science), of course this particularly question could be very easily solved in bash (but for some reason it ain't trendy).


---

