<p>A single MLP layer with no activation function is a linear function, as you already seem to know, and hence trivially equivalent to a projection.  It is equivalent to a random projection if you choose the entries in the matrix randomly from the appropriate distribution.</p>
<p>Normally a single-layer MLP is understood to include an activation.  That is not equivalent to a random projection.</p>
<p>The value of this equivalence is unclear.  If you want a random projection, it seems clearer to simply use a random projection, without needing to introduce a MLP.</p>

CC BY-SA 4.0
2023-07-11T02:32:14.107Z
Thanks for the response. I'm asking because I want to use a random projection in a Tensorflow project, and as far as I can tell, TF doesn't have it yet.   As for an MLP without an activation, I've seen some projects use it for linear operations [(like here for example)](https://github.com/oozdenizci/MMIDimReduction/blob/27b268e1e30d0653073f989eb217203296c0265d/MMIDimReduction.py#L147).
Liam F-A
CC BY-SA 4.0
2023-07-11T01:53:54.027Z
2023-07-11T01:53:54.027Z
D.W.
<p>I am just hoping to confirm my hypothesis, that a single MLP (untrained and randomly initialized) can be used for <a href="https://scikit-learn.org/stable/modules/random_projection.html" rel="nofollow noreferrer">random projection</a> for dimensionality reduction.</p>
<p>If a random MLP layer with no activation is:</p>
<p><span class="math-container">$$Y = X \ W  $$</span></p>
<p>is there really any difference to the random projection method, especially if its weights are sampled from a distribution with a variance following the <a href="https://scikit-learn.org/stable/modules/random_projection.html#the-johnson-lindenstrauss-lemma" rel="nofollow noreferrer">Johnson–Lindenstrauss lemma
</a> ?</p>

<machine-learning><st.statistics><ai.artificial-intel>
Is a Single Linear MLP Equivalent to a Random Projection
Liam F-A
# Is a Single Linear MLP Equivalent to a Random Projection

(Asked by: username_0 on 2023-07-10)

I am just hoping to confirm my hypothesis, that a single MLP (untrained and randomly initialized) can be used for [random projection](https://scikit-learn.org/stable/modules/random_projection.html) for dimensionality reduction.

If a random MLP layer with no activation is:

$$Y = X \ W $$

is there really any difference to the random projection method, especially if its weights are sampled from a distribution with a variance following the [Johnson–Lindenstrauss lemma](https://scikit-learn.org/stable/modules/random_projection.html#the-johnson-lindenstrauss-lemma) ?

**Tags:** machine-learning, st.statistics, ai.artificial-intel

---

## 1 answers

### Answer by username_1 on 2023-07-11. Score: 1

A single MLP layer with no activation function is a linear function, as you already seem to know, and hence trivially equivalent to a projection. It is equivalent to a random projection if you choose the entries in the matrix randomly from the appropriate distribution.

Normally a single-layer MLP is understood to include an activation. That is not equivalent to a random projection.

The value of this equivalence is unclear. If you want a random projection, it seems clearer to simply use a random projection, without needing to introduce a MLP.

> **Comments:**
> **username_0**: Thanks for the response. I'm asking because I want to use a random projection in a Tensorflow project, and as far as I can tell, TF doesn't have it yet.   As for an MLP without an activation, I've seen some projects use it for linear operations [(like here for example)](https://github.com/oozdenizci/MMIDimReduction/blob/27b268e1e30d0653073f989eb217203296c0265d/MMIDimReduction.py#L147).  

---

