<p>Another option is to use <code>list.files</code> and <code>grep</code> to select only the <code>.txt</code> in your directory. A <code>for</code> loop can read-aggregate and save the files one at the time.</p>

<pre><code>all.files &lt;- list.files("/location folder")
txt.files &lt;- grep(".txt",all.files,value=T)

for(i in txt.files){
  d &lt;- read.table(paste("/location folder",i,sep=""),header=F)
  x &lt;- aggregate(d[, 2], list(d$V3), median)
  write.table(x, paste0(i,".output.txt"), quote=FALSE, row.names=TRUE)
}
</code></pre>

CC BY-SA 4.0
2019-07-02T15:07:02.573Z
Why not just `list.files(pattern = "*.txt")`? The `grep` is not required.
Ram RS
CC BY-SA 4.0
2019-07-02T15:07:52.017Z
And looping when vectorized operations are available is de-optimization.
Ram RS
CC BY-SA 4.0
2019-07-02T15:10:50.797Z
And there's some inconsistent usage: `paste(..., sep="")` on line 5 and `paste0()` on line 7.
Ram RS
CC BY-SA 4.0
2019-07-02T15:21:42.803Z
considering the nature of the question I thought that it was more important to show possibilities rather than one single point of view. `paste` and `paste0` achieve the same but in different ways (e.g. `sep=""`).  I tend to split commands involving `list.files` and `grep` as it makes easier to read and to reuse. The content of `all.files` is hard to miss. Finally the usage of the `apply`family is not easy to grasp for new user, while loops are much intuitive. Furthermore, `apply` is not always better than a nice `for`, especially in an example like this.
fra
CC BY-SA 4.0
2019-07-02T15:32:25.807Z
Showing possibilities is great! However, I don't think a loop would perform as well as or better than vectorized operations. While a loop is easier for people that know basic programming, the `apply` family solves a lot of problems, especially when there is a learning opportunity such as this one, where the task is well defined and how apply works can be easily understood.
Ram RS
CC BY-SA 4.0
2019-07-03T14:29:13.357Z
Just encountered this particular gotcha, so we might have to be careful about using `T`/`F` instead of `TRUE`/`FALSE` as well: https://www.johndcook.com/blog/r_language_for_programmers/#assignment
Ram RS
CC BY-SA 4.0
2019-07-02T14:21:56.647Z
2019-07-02T14:21:56.647Z
fra
<p>Please do not use <code>setwd</code> in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.</p>

<p>Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a <code>list.files(pattern = "&lt;pattern_here&gt;")</code>. Read this file as a <code>data.frame</code> and use it with an <code>apply</code>, so a function processes the df row-by-row. Let's say the file (named <code>metadata.txt</code>) is:</p>

<pre><code>in_file out_file
tgf.txt tgf.output.txt
abc.txt abc.filtered.txt
xyz.txt xyz.randomname.txt
</code></pre>

<p>Define a function called <code>process_row()</code> in the Rscript:</p>

<pre><code>process_row &lt;- function(r) {
    d &lt;- read.table(r[[1]], header = FALSE);
    x &lt;- aggregate(d[, 2], list(d$V3), median);
    write.table(x, r[[2]], quote=FALSE, row.names=TRUE);
}
</code></pre>

<p>And do the actual work:</p>

<pre><code>meta_data &lt;- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);
apply(meta_data, 1, process_row(r))
# You don't need a q() in an Rscript because it exits after the last line of code anyway.
</code></pre>

CC BY-SA 4.0
2019-07-02T09:57:28.590Z
I tried the above script but it is giving me following error

Error in read.table(r[[1]], header = FALSE) : 
  'file' must be a character string or connection
Called from: read.table(r[[1]], header = FALSE)
Megha
CC BY-SA 4.0
2019-07-02T13:40:39.020Z
What is the content of `./metadata.txt`, which is in the working directory where you're running R?
Ram RS
CC BY-SA 4.0
2019-07-03T14:45:50.610Z
It worked. Thank you
Megha
CC BY-SA 4.0
2019-07-03T17:42:07.027Z
Please accept the answer if it worked. Thank you
Ram RS
CC BY-SA 4.0
2019-07-01T16:14:38.640Z
2019-07-02T15:11:33.410Z
2019-07-02T15:11:33.410Z
Ram RS
<p>I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files</p>

<pre><code> #!/usr/bin/R
 setwd("/location folder")

 d &lt;- read.table("tgf.txt", header=FALSE)

 x &lt;- aggregate(d[, 2], list(d$V3), median)

 write.table(x, "tgf.output.txt", quote=FALSE, row.names=TRUE)

 q()
</code></pre>

<r>
Run Rscript for multiple samples (files)
# Run Rscript for multiple samples (files)

(Asked by: username_0 on 2019-07-01)

I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files
 #!/usr/bin/R
 setwd("/location folder")

 d <- read.table("tgf.txt", header=FALSE)

 x <- aggregate(d[, 2], list(d$V3), median)

 write.table(x, "tgf.output.txt", quote=FALSE, row.names=TRUE)

 q()

**Tags:** r

---

## 2 answers

### Answer by username_1 on 2019-07-02. Score: 1 | Accepted answer

Another option is to use list.files and grep to select only the .txt in your directory. A for loop can read-aggregate and save the files one at the time.
all.files <- list.files("/location folder")
txt.files <- grep(".txt",all.files,value=T)

for(i in txt.files){
  d <- read.table(paste("/location folder",i,sep=""),header=F)
  x <- aggregate(d[, 2], list(d$V3), median)
  write.table(x, paste0(i,".output.txt"), quote=FALSE, row.names=TRUE)
}

> **Comments:**
> **username_2**: Why not just `list.files(pattern = "*.txt")`? The `grep` is not required.  
> **username_2**: And looping when vectorized operations are available is de-optimization.  
> **username_2**: And there's some inconsistent usage: `paste(..., sep="")` on line 5 and `paste0()` on line 7.  
> **username_1**: considering the nature of the question I thought that it was more important to show possibilities rather than one single point of view. `paste` and `paste0` achieve the same but in different ways (e.g. `sep=""`).  I tend to split commands involving `list.files` and `grep` as it makes easier to read and to reuse. The content of `all.files` is hard to miss. Finally the usage of the `apply`family is not easy to grasp for new user, while loops are much intuitive. Furthermore, `apply` is not always better than a nice `for`, especially in an example like this.  
> **username_2**: Showing possibilities is great! However, I don't think a loop would perform as well as or better than vectorized operations. While a loop is easier for people that know basic programming, the `apply` family solves a lot of problems, especially when there is a learning opportunity such as this one, where the task is well defined and how apply works can be easily understood.  
> **username_2**: Just encountered this particular gotcha, so we might have to be careful about using `T`/`F` instead of `TRUE`/`FALSE` as well: https://www.johndcook.com/blog/r_language_for_programmers/#assignment  

---

### Answer by username_2 on 2019-07-01. Score: 3

Please do not use setwd in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.
Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a list.files(pattern = ""). Read this file as a data.frame and use it with an apply, so a function processes the df row-by-row. Let's say the file (named metadata.txt) is:
in_file out_file
tgf.txt tgf.output.txt
abc.txt abc.filtered.txt
xyz.txt xyz.randomname.txt

Define a function called process_row() in the Rscript:
process_row <- function(r) {
    d <- read.table(r[[1]], header = FALSE);
    x <- aggregate(d[, 2], list(d$V3), median);
    write.table(x, r[[2]], quote=FALSE, row.names=TRUE);
}

And do the actual work:
meta_data <- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);
apply(meta_data, 1, process_row(r))
# You don't need a q() in an Rscript because it exits after the last line of code anyway.

> **Comments:**
> **username_0**: I tried the above script but it is giving me following error  Error in read.table(r[[1]], header = FALSE) :    'file' must be a character string or connection Called from: read.table(r[[1]], header = FALSE)  
> **username_2**: What is the content of `./metadata.txt`, which is in the working directory where you're running R?  
> **username_0**: It worked. Thank you  
> **username_2**: Please accept the answer if it worked. Thank you  

---

