<p>Go to the common parent folder under which all FASTQ.GZ files reside. They could be in sub-folders under the folder, but all of them should have the folder you're in as a common parent folder. From such a folder, you should be able to run a <code>find</code></p>

<pre><code>find . -name "*.fastq.gz"
</code></pre>

<p>to get your FASTQ.GZ files. Now, there are surely better ways of doing this but here's one straightforward way.</p>

<p>Let's say you need to extract the FASTQs to <code>~/target/dir/</code>, you'd do something like:</p>

<pre><code>for f in <span class="math-container">$(find . -name "*.fastq.gz")
do
    tgt_file="~/target/dir/$</span>{f%.*}" #removed .gz from the name
    gzip -dc <span class="math-container">$f &gt;$</span>tgt_file #pigz -dc would be better if you have pigz
done
</code></pre>

<p>EDIT: OP has clarified that they wish to copy the files as such, not decompress them.</p>

<p>There are two ways to achieve this goal - you can either make physical copies of the files (thereby doubling the storage space you're currently using for them), or you can create symbolic links to the files (which are like shortcuts pointing to the actual files). I'll show you both ways here:</p>

<p>Let's assume the files are all under <code>/fastq/files</code></p>

<pre><code>find /fastq/files/ -name "*.fastq.gz" -exec cp {} ~/target/dir/ \; #This creates physical copies

cd ~/target/dir
find /fastq/files/ -name "*.fastq.gz" | xargs -I v_f ln -s v_f #These lines create symbolic links that point to the original files. Do not delete/move/rename the original files or these will become invalid.
</code></pre>

CC BY-SA 4.0
2019-07-01T15:19:41.057Z
Dear Ram, I want all subdirectory fastq.gz files into a single folder but files should be in fastq.gz format only, I don't want to decompress the files.
bioinfonext
CC BY-SA 4.0
2019-07-01T16:08:13.527Z
I've updated my answer. Use symbolic links instead of copying the files. As long as you don't move the original files or rename them (or any of their parent directories), your links will work fine and take much less space than if you copied the actual files.
Ram RS
CC BY-SA 4.0
2019-07-02T02:21:48.293Z
@Dr.Yogesh Please accept the answer if it worked for you.
Ram RS
CC BY-SA 4.0
2019-07-01T14:22:29.003Z
2019-07-01T16:54:24.860Z
2019-07-01T16:54:24.860Z
Ram RS
<p>I have all file names for each paired-end fastq pair as shown below. Could you please suggest how I can extract all fastq.gz files in a single folder?</p>

<p>sample name:</p>

<pre><code>188833984]$ ls

Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc
Leaf_T1_FD_R2_L001-ds.7db8eb7e3426486db549426601b3a0bd
Leaf_T1_FD_R3_L001-ds.147177ecc03a46ccbbce3162d1185a0a
</code></pre>

<p>pair end file within sample name Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc</p>

<pre><code>188833984]$ cd Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc

Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc]$ ls
Leaf-T1-FD-R10_S73_L001_R1_001.fastq.gz
Leaf-T1-FD-R10_S73_L001_R2_001.fastq.gz
</code></pre>

<p>I need all in fastq.gz format but the problem is that each sample, pair end R1 and R2 fastq reads in a single folder like this: and I need to extract all samples files in a single folder</p>

CC BY-SA 4.0
2019-07-01T12:15:05.487Z
Can you please [edit] your question and clarify. What do you mean by "extract all fastq files"? Do you just want to decompress (go from `fastq.gz` to `fastq`)? What is the final outcome you are expecting from the example you give?
terdon
CC BY-SA 4.0
2019-07-01T14:18:13.803Z
I think OP wants to decompress `fastq.gz` files found in various subdirectories to `fastq` files in a single location. OP should probably decompress them where they are and create soft-links to them in their directory of choice.
Ram RS
CC BY-SA 4.0
2019-07-01T14:20:09.923Z
@Dr.Yogesh, Do you want to copy/move all files to one folder or decompress them (as Ram RS is suggesting)?
benn
CC BY-SA 4.0
2019-07-01T14:29:38.903Z
"this file name has R1 and R1 fastq.gz": what does this mean? Did you mean to write R1 and R*2*?  You show a directory, `Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc` with two files, an `R1` and an `R2`, so what is the problem? The files seem to already be in separate directories. Can you please [edit] and show us what the final result you want is?
terdon
<bash><awk>
How to extract all pair end fastq.gz files from multiple subdirectory to a single folder
# How to extract all pair end fastq.gz files from multiple subdirectory to a single folder

(Asked by: username_0 on 2019-07-01)

I have all file names for each paired-end fastq pair as shown below. Could you please suggest how I can extract all fastq.gz files in a single folder?
sample name:
188833984]$ ls

Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc
Leaf_T1_FD_R2_L001-ds.7db8eb7e3426486db549426601b3a0bd
Leaf_T1_FD_R3_L001-ds.147177ecc03a46ccbbce3162d1185a0a

pair end file within sample name Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc
188833984]$ cd Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc

Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc]$ ls
Leaf-T1-FD-R10_S73_L001_R1_001.fastq.gz
Leaf-T1-FD-R10_S73_L001_R2_001.fastq.gz

I need all in fastq.gz format but the problem is that each sample, pair end R1 and R2 fastq reads in a single folder like this: and I need to extract all samples files in a single folder

**Tags:** bash, awk
> **Comments:**
> **username_1**: Can you please [edit] your question and clarify. What do you mean by "extract all fastq files"? Do you just want to decompress (go from `fastq.gz` to `fastq`)? What is the final outcome you are expecting from the example you give?  
> **username_2**: I think OP wants to decompress `fastq.gz` files found in various subdirectories to `fastq` files in a single location. OP should probably decompress them where they are and create soft-links to them in their directory of choice.  
> **username_3**: @Dr.Yogesh, Do you want to copy/move all files to one folder or decompress them (as Ram RS is suggesting)?  
> **username_1**: "this file name has R1 and R1 fastq.gz": what does this mean? Did you mean to write R1 and R*2*?  You show a directory, `Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc` with two files, an `R1` and an `R2`, so what is the problem? The files seem to already be in separate directories. Can you please [edit] and show us what the final result you want is?  

---

## 1 answers

### Answer by username_2 on 2019-07-01. Score: 3 | Accepted answer

Go to the common parent folder under which all FASTQ.GZ files reside. They could be in sub-folders under the folder, but all of them should have the folder you're in as a common parent folder. From such a folder, you should be able to run a find
find . -name "*.fastq.gz"

to get your FASTQ.GZ files. Now, there are surely better ways of doing this but here's one straightforward way.
Let's say you need to extract the FASTQs to ~/target/dir/, you'd do something like:
for f in $(find . -name "*.fastq.gz")
do
    tgt_file="~/target/dir/${f%.*}" #removed .gz from the name
    gzip -dc $f >$tgt_file #pigz -dc would be better if you have pigz
done

EDIT: OP has clarified that they wish to copy the files as such, not decompress them.
There are two ways to achieve this goal - you can either make physical copies of the files (thereby doubling the storage space you're currently using for them), or you can create symbolic links to the files (which are like shortcuts pointing to the actual files). I'll show you both ways here:
Let's assume the files are all under /fastq/files
find /fastq/files/ -name "*.fastq.gz" -exec cp {} ~/target/dir/ \; #This creates physical copies

cd ~/target/dir
find /fastq/files/ -name "*.fastq.gz" | xargs -I v_f ln -s v_f #These lines create symbolic links that point to the original files. Do not delete/move/rename the original files or these will become invalid.

> **Comments:**
> **username_0**: Dear Ram, I want all subdirectory fastq.gz files into a single folder but files should be in fastq.gz format only, I don't want to decompress the files.  
> **username_2**: I've updated my answer. Use symbolic links instead of copying the files. As long as you don't move the original files or rename them (or any of their parent directories), your links will work fine and take much less space than if you copied the actual files.  
> **username_2**: @Dr.Yogesh Please accept the answer if it worked for you.  

---

