<p>I can see what you're wanting to do, I can see bugs but the key bit of output is missing (below). It seems a bit weird to me the nested loop in context, but thats fine if you're happy.</p>
<p>The critical step is this</p>
<pre><code>df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
</code></pre>
<p>What I'm concerned about is the <code>pd.concat</code>. Are you sure <code>pd.concat</code> is the right command here? I thought <code>m</code> is one data table, e.g. mutation and <code>g</code> is another e.g. expression. <code>pd.concat</code> isn't going to work here, are you sure this isn't a <code>merge</code> or <code>join</code> command?</p>
<p>Anyway, the fitting is complaining the <code>.shape()</code> is wrong. Needs reshaping? Thats the error. Printing <code>df</code> at that point would be needed.</p>
<blockquote>
<p>ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)</p>
</blockquote>
<p>To summarise normally the error you have received is about re-shaping the data. I've not put forward that solution here because,</p>
<ul>
<li>I am not sure what the data represents</li>
<li>If I was doing this I wouldn't use <code>pd.concat</code></li>
</ul>
<p>It seems a lot of columns for a 1-way ANOVA, but could be right, I've not done 1-way ANOVA across 10 columns.</p>
<p><em>Bug 1</em></p>
<pre><code>m = pd.read_csv(&quot;mut.csv&quot;, sep=&quot;,&quot;)
m=pd.DataFrame(m)
m.info()
g = pd.read_csv(&quot;expr1.csv&quot;, sep=&quot;,&quot;)
g=pd.DataFrame(g)
</code></pre>
<p>This should simply be:</p>
<pre><code>m = pd.read_csv(&quot;mut.csv&quot;)
g = pd.read_csv(&quot;expr1.csv&quot;)
</code></pre>
<p><em>Bug 2?</em></p>
<pre><code>dfs = pd.DataFrame({})
...
dfs = dfs.append(aov_table)
</code></pre>
<p>This will work given <code>aov_table</code> output is a pandas dataframe (I'm not at all sure it is). You need <code>aov_table.info()</code> to be sure. You can always place it in a dataframe however.</p>

CC BY-SA 4.0
2023-09-27T18:35:08.180Z
2023-09-27T23:01:10.543Z
2023-09-27T23:01:10.543Z
M__
<p>I have mutation status (<code>0</code>,<code>1</code>) for ten genes in ten cancer 3D models</p>
<p>I have also gene expression in these 3D models, like</p>
<pre class="lang-r prettyprint-override"><code>expr1 &lt;- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = &quot;data.frame&quot;, row.names = c(&quot;ModelM&quot;, &quot;ModelO&quot;, &quot;ModelS&quot;, &quot;ModelA&quot;, &quot;ModelI&quot;, &quot;ModelC&quot;, &quot;ModelT&quot;, &quot;ModelH&quot;, &quot;ModelE&quot;, &quot;ModelR&quot;, &quot;ModelA2&quot;, &quot;ModelP&quot;, &quot;ModelE2&quot;, &quot;ModelU&quot;, &quot;ModelT2&quot;, &quot;ModelC2&quot;, &quot;ModelS2&quot;, &quot;ModelX&quot;))
mut &lt;- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = &quot;data.frame&quot;, row.names = c(&quot;Gene1_mut&quot;, &quot;Gene2_mut&quot;, &quot;Gene3_mut&quot;, &quot;Gene4_mut&quot;, &quot;Gene5_mut&quot;, &quot;Gene6_mut&quot;, &quot;Gene7_mut&quot;, &quot;Gene8_mut&quot;, &quot;Gene9_mut&quot;, &quot;Gene10_mut&quot;))
</code></pre>
<p><code>mut.csv</code> and <code>expr1.csv</code> exported for use in Python:</p>
<pre class="lang-r prettyprint-override"><code>write.csv(mut, &quot;mut.csv&quot;, quote = FALSE, row.names = FALSE)
write.csv(expr1[1:10,], &quot;expr1.csv&quot;, quote = FALSE, row.names = FALSE)
</code></pre>
<p>I want to do a one-way ANOVA in Python so that calculate P-VALUE for each pair of mutation-expression in each pair of gene-model</p>
<p>This code returns error</p>
<pre class="lang-py prettyprint-override"><code>import os
import pandas as pd
from statsmodels.formula.api import ols
import statsmodels.api as sm
m = pd.read_csv(&quot;mut.csv&quot;, sep=&quot;,&quot;)
m=pd.DataFrame(m)
m.info()
g = pd.read_csv(&quot;expr1.csv&quot;, sep=&quot;,&quot;)
g=pd.DataFrame(g)

lst = list(g.columns)
m_lst = list(m.columns)

dfs = pd.DataFrame({})
for Mut in m_lst:
    for Model in lst:
        df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
        df = df.rename(columns={Mut: &quot;mut&quot;, Model: &quot;g&quot;})
        model = ols('g ~ C(mut)', data=df).fit()
ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)

        aov_table = sm.stats.anova_lm(model, typ=2)
        aov_table[['Model']] = Model
        aov_table[['Mut']] = Mut
        dfs = dfs.append(aov_table)
</code></pre>
<p>The ideal output is something like</p>
<pre class="lang-py prettyprint-override"><code>with pd.option_context('display.max_rows', None, 'display.max_columns', None):
    print(dfs)
            sum_sq   df         F    PR(&gt;F)     Model      Mut
C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM
Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM
C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM
Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM
C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO
Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO
</code></pre>

<python><for-loop>
For loop in Python goes wrong
# For loop in Python goes wrong

(Asked by: username_0 on 2023-09-27)

I have mutation status (0,1) for ten genes in ten cancer 3D models
I have also gene expression in these 3D models, like
expr1 <- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = "data.frame", row.names = c("ModelM", "ModelO", "ModelS", "ModelA", "ModelI", "ModelC", "ModelT", "ModelH", "ModelE", "ModelR", "ModelA2", "ModelP", "ModelE2", "ModelU", "ModelT2", "ModelC2", "ModelS2", "ModelX"))
mut <- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = "data.frame", row.names = c("Gene1_mut", "Gene2_mut", "Gene3_mut", "Gene4_mut", "Gene5_mut", "Gene6_mut", "Gene7_mut", "Gene8_mut", "Gene9_mut", "Gene10_mut"))

mut.csv and expr1.csv exported for use in Python:
write.csv(mut, "mut.csv", quote = FALSE, row.names = FALSE)
write.csv(expr1[1:10,], "expr1.csv", quote = FALSE, row.names = FALSE)

I want to do a one-way ANOVA in Python so that calculate P-VALUE for each pair of mutation-expression in each pair of gene-model
This code returns error
import os
import pandas as pd
from statsmodels.formula.api import ols
import statsmodels.api as sm
m = pd.read_csv("mut.csv", sep=",")
m=pd.DataFrame(m)
m.info()
g = pd.read_csv("expr1.csv", sep=",")
g=pd.DataFrame(g)

lst = list(g.columns)
m_lst = list(m.columns)

dfs = pd.DataFrame({})
for Mut in m_lst:
    for Model in lst:
        df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)
        df = df.rename(columns={Mut: "mut", Model: "g"})
        model = ols('g ~ C(mut)', data=df).fit()
ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)

        aov_table = sm.stats.anova_lm(model, typ=2)
        aov_table[['Model']] = Model
        aov_table[['Mut']] = Mut
        dfs = dfs.append(aov_table)

The ideal output is something like
with pd.option_context('display.max_rows', None, 'display.max_columns', None):
    print(dfs)
            sum_sq   df         F    PR(>F)     Model      Mut
C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM
Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM
C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM
Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM
C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO
Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO

**Tags:** python, for-loop

---

## 1 answers

### Answer by username_1 on 2023-09-27. Score: 1

I can see what you're wanting to do, I can see bugs but the key bit of output is missing (below). It seems a bit weird to me the nested loop in context, but thats fine if you're happy.
The critical step is this
df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)

What I'm concerned about is the pd.concat. Are you sure pd.concat is the right command here? I thought m is one data table, e.g. mutation and g is another e.g. expression. pd.concat isn't going to work here, are you sure this isn't a merge or join command?
Anyway, the fitting is complaining the .shape() is wrong. Needs reshaping? Thats the error. Printing df at that point would be needed.

ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)

To summarise normally the error you have received is about re-shaping the data. I've not put forward that solution here because,

I am not sure what the data represents
If I was doing this I wouldn't use pd.concat

It seems a lot of columns for a 1-way ANOVA, but could be right, I've not done 1-way ANOVA across 10 columns.
Bug 1
m = pd.read_csv("mut.csv", sep=",")
m=pd.DataFrame(m)
m.info()
g = pd.read_csv("expr1.csv", sep=",")
g=pd.DataFrame(g)

This should simply be:
m = pd.read_csv("mut.csv")
g = pd.read_csv("expr1.csv")

Bug 2?
dfs = pd.DataFrame({})
...
dfs = dfs.append(aov_table)

This will work given aov_table output is a pandas dataframe (I'm not at all sure it is). You need aov_table.info() to be sure. You can always place it in a dataframe however.


---

