<p>Converting your <code>data.frame</code> to a <code>matrix</code> (and then removing the <code>data.frame</code>) will often free up enough memory that you won't run into this. Note that a <code>matrix</code> is more memory efficient than a <code>data.frame</code> and you're requiring <code>Rtsne()</code> to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).</p>

<p>For what it's worth, it's never been entirely clear to me what the interaction is between a <code>data.frame</code> and the pointer protection stack, but it's often the case that this solves this sort of error.</p>

CC BY-SA 3.0
2017-09-12T15:02:42.103Z
`data.frames` are lists of column vectors, and every one of these columns has a pointer that may need protecting. With a huge number of columns, I can easily see how this might blow the protect stack. Dense matrices are contiguous memory, they only need a single pointer.
Konrad Rudolph
CC BY-SA 3.0
2017-09-12T16:23:50.280Z
I'd considered that, but my reading never clarified if it's really due to pointers for the columns or something else. It's unfortunate how little is mentioned about that in the documentation.
Devon Ryan
CC BY-SA 3.0
2017-09-04T12:07:15.483Z
2017-09-04T12:07:15.483Z
Devon Ryan
<p>I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.</p>

<p>Here is the code that I'm trying to run.</p>

<pre><code># t-SNE implementation
library(Rtsne)

set.seed(1)

for(i in 1:15){

  tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)

  colors = rainbow(length(unique(data.T$classes)))
  names(colors) = unique(data.T$classes)
  plot(tsne$Y, t='n', main="tsne")
  text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])

  readline(prompt="Press [enter] to continue")
}
</code></pre>

<p>Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.</p>

<p>The think here is that when I execute this script, R returns me this error:</p>

<blockquote>
  <p>Error: protect(): protection stack overflow</p>
</blockquote>

<p>Should I change the <code>--max-pp-size</code> or it's a bug in Rtsne package?</p>

<p>Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.</p>

<p>For the configuration of the Rtsne function I read <a href="https://distill.pub/2016/misread-tsne/" rel="noreferrer">this article</a></p>

<p>Any other suggestion on the implementation is welcomed.</p>

<r><visualization><microarray><clustering>
Using the t-SNE algorithm on microarray data + an error bonus
# Using the t-SNE algorithm on microarray data + an error bonus

(Asked by: username_0 on 2017-09-02)

I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.
Here is the code that I'm trying to run.
# t-SNE implementation
library(Rtsne)

set.seed(1)

for(i in 1:15){

  tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)

  colors = rainbow(length(unique(data.T$classes)))
  names(colors) = unique(data.T$classes)
  plot(tsne$Y, t='n', main="tsne")
  text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])

  readline(prompt="Press [enter] to continue")
}

Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.
The think here is that when I execute this script, R returns me this error:

Error: protect(): protection stack overflow

Should I change the --max-pp-size or it's a bug in Rtsne package?
Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.
For the configuration of the Rtsne function I read this article
Any other suggestion on the implementation is welcomed.

**Tags:** r, visualization, microarray, clustering

---

## 1 answers

### Answer by username_1 on 2017-09-04. Score: 13 | Accepted answer

Converting your `data.frame` to a `matrix` (and then removing the `data.frame`) will often free up enough memory that you won't run into this. Note that a `matrix` is more memory efficient than a `data.frame` and you're requiring `Rtsne()` to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).

For what it's worth, it's never been entirely clear to me what the interaction is between a `data.frame` and the pointer protection stack, but it's often the case that this solves this sort of error.

> **Comments:**
> **username_2**: `data.frames` are lists of column vectors, and every one of these columns has a pointer that may need protecting. With a huge number of columns, I can easily see how this might blow the protect stack. Dense matrices are contiguous memory, they only need a single pointer.  
> **username_1**: I'd considered that, but my reading never clarified if it's really due to pointers for the columns or something else. It's unfortunate how little is mentioned about that in the documentation.  

---

