<p>Second viewing of the question from what I can see -0.22 as a coefficient of origin is a strong negative association, so yeah it has a major impact. Its not how I would have done it, but that looks to be the result.</p>
<hr />
<p>First viewing,</p>
<p>I'm going to throw my hat in here. We don't know what 'origin' is about, anyway just throw everything, i.e. each substrate and the origin into the same regression calculation. Check for a low-residual and preferably do a Q-Q plot, transform your data it this doesn't look good.</p>
<p>The key and the thing you are missing is your regression weights, without that I couldn't say very much. If the regression weight is near zero for 'origin' then it has zero impact. If the regression weight of 'origin' is positively greater than everything else ... I assume there are skewed distributions of 'substrates' between the 'origins'. If the regression weight of 'origin' is negative but still greater than all other regression weights then it is adversely affecting the 'value' you are seeking.</p>
<p>I don't know the experiment, the biological system or really the 'substrate' assays, so I can't comment any further.</p>
<p>The two issues I have are:</p>
<ol>
<li>Doing an ANOVA on the output of a regression analysis doesn't make much sense to me. It is not something I would do, nor something in ML or GLM I've encountered.</li>
<li>Are you doing pairwise substrate/origin calculations? I presue not, but just in case this not how GLM works.</li>
</ol>

CC BY-SA 4.0
2020-06-29T07:31:18.023Z
oh I was under the impression `aov(summary(glm_model))` was a pretty common practice as it nicely summarises down to a variable level? Where do I find the regression weights?
Lamma
CC BY-SA 4.0
2020-06-29T07:36:41.693Z
When transforming the data do I have to transform both variables or can I transform just one of them until they fit the same distribution? and given that I am using a negative binomial glm does a uniform distribution between the variables matter as much?

I am unsure if there is a skewed distribution of variables between origins, that is what I am looking to find out really.
Lamma
CC BY-SA 4.0
2020-06-29T07:36:45.813Z
hi @Michael, very nice points here, yes the term anova is quite misleading sometimes. Normally people associate associate with the classical analysis of variance. In more "modern" uses, it can refer to the breakdown of deviance in a model, https://stats.stackexchange.com/questions/289226/what-are-the-differences-between-anovas-and-glms
StupidWolf
CC BY-SA 4.0
2020-06-29T07:38:27.387Z
It is like a log likelihood ratio test essentially, the "F" statistic instead of Chi-sq is used.
StupidWolf
CC BY-SA 4.0
2020-06-29T07:39:58.853Z
@StupidWolf So it is fine to `aov(summary(glm_model))` to get a nice overview of the model? And by pairwise substrate/origin calculations does you mean doing the model on a subsatrate of the dataframe for each substrate @michael?
Lamma
CC BY-SA 4.0
2020-06-27T12:46:40.800Z
2020-06-29T00:35:18.887Z
2020-06-29T00:35:18.887Z
M__
<p>There is no better method, it's a matter of what you want to test or what is your question.</p>
<p>Using the <code>anova()</code> or <code>aov()</code>, test the terms collectively. For example, in your example with Substrate, the null hypothesis is that the coefficients are all zero, meaning <code>cellulose =0, mannan =0 , ....</code></p>
<p>If the question is, &quot;do the isolate samples have a higher value than origin samples?&quot;, then you can use your first model, where <code>free</code> is set as the reference and you test whether the effect of <code>isolate</code> is non-zero. Likewise you can do this for substrate and set of them as your reference. You can also do other pairwise comparisons using this model.</p>
<p>If the question is, &quot;does origin have a significant effect on value, after controlling for substrate?&quot;, then you can use your second model.</p>

CC BY-SA 4.0
2020-06-29T07:20:25.070Z
Would I control for substrate by forming the model as such: value ~ origin + substrate ?
Lamma
CC BY-SA 4.0
2020-06-29T07:21:56.630Z
yes you do. you are giving every substrate a meaning  and on top of that estimating the effect of origin
StupidWolf
CC BY-SA 4.0
2020-06-29T07:24:53.787Z
And does the order in which I give the variable to the model, i.e. origin before substrate, indicate some sort of predefined importance to the model?
Lamma
CC BY-SA 4.0
2020-06-29T07:28:18.840Z
it doesn't define the importance, you can put it in any order. However if they are correlated, most of the variance will go to the first. I don't think this will apply in your case
StupidWolf
CC BY-SA 4.0
2020-06-29T07:32:26.997Z
see https://cran.r-project.org/doc/contrib/Faraway-PRA.pdf blocking design. A good ieda to read this
StupidWolf
CC BY-SA 4.0
2020-06-29T07:33:01.713Z
Thank you I will look at this :)
Lamma
CC BY-SA 4.0
2020-06-27T12:06:33.487Z
2020-06-27T12:06:33.487Z
StupidWolf
<p>How would one determine the significance of a variable in a glm model?</p>
<p>If I, for example, have a dataframe like the one seen below, how would I determine if the origin of the sample has a significant effect on the value? (this is the number of enzymes capable of degrading the substrate of that matter)</p>
<pre><code>Substrate    variable value origin
cellulose       M09    8    free
mannan          M12    2    free
glycogen        M65    2    free
chitin          M87    4    free
cellulose       M90    2    isolate
manan           M78    1    isolate
glycogen        M21    4    isolate
chitin          M21    1    isolate
</code></pre>
<p>So far I have tried:</p>
<pre class="lang-r prettyprint-override"><code>mcomp = glm.nb(value ~ origin, data = my_data)

summary(mcomp)
Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-0.9625  -0.9047  -0.9047   0.1212   3.5232  

Coefficients:
              Estimate Std. Error z value Pr(&gt;|z|)   
(Intercept)   -0.01657    0.06571  -0.252  0.80097   
originisolate -0.21911    0.08180  -2.679  0.00739 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for Negative Binomial(0.3418) family taken to be 1)

    Null deviance: 2053.5  on 2679  degrees of freedom
Residual deviance: 2046.3  on 2678  degrees of freedom
AIC: 6517.5

Number of Fisher Scoring iterations: 1


              Theta:  0.3418 
          Std. Err.:  0.0186 

 2 x log-likelihood:  -6511.4590 
</code></pre>
<p>So free becomes the intercept and then isolate if significantly different from that. Does this mean Origin has a significant effect on the value?</p>
<p>Would the better approach be to do the following?:</p>
<pre class="lang-r prettyprint-override"><code>mcomp = glm.nb(value ~ origin + Substrate, data = comb_data) 
summary(aov(mcomp))
              Df Sum Sq Mean Sq F value Pr(&gt;F)    
origin         1     23   22.55   6.612 0.0102 *  
Substrate     44   1445   32.84   9.631 &lt;2e-16 ***
Residuals   2634   8981    3.41                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 
</code></pre>
<p>This shows me that origin and substrate have an effect on value if I understand correctly?</p>

<modelling><linear-regression><statsmodels>
Determining significance of a variable in a glm model
# Determining significance of a variable in a glm model

(Asked by: username_0 on 2020-06-26)

How would one determine the significance of a variable in a glm model?
If I, for example, have a dataframe like the one seen below, how would I determine if the origin of the sample has a significant effect on the value? (this is the number of enzymes capable of degrading the substrate of that matter)
Substrate    variable value origin
cellulose       M09    8    free
mannan          M12    2    free
glycogen        M65    2    free
chitin          M87    4    free
cellulose       M90    2    isolate
manan           M78    1    isolate
glycogen        M21    4    isolate
chitin          M21    1    isolate

So far I have tried:
mcomp = glm.nb(value ~ origin, data = my_data)

summary(mcomp)
Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-0.9625  -0.9047  -0.9047   0.1212   3.5232  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)   
(Intercept)   -0.01657    0.06571  -0.252  0.80097   
originisolate -0.21911    0.08180  -2.679  0.00739 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for Negative Binomial(0.3418) family taken to be 1)

    Null deviance: 2053.5  on 2679  degrees of freedom
Residual deviance: 2046.3  on 2678  degrees of freedom
AIC: 6517.5

Number of Fisher Scoring iterations: 1


              Theta:  0.3418 
          Std. Err.:  0.0186 

 2 x log-likelihood:  -6511.4590 

So free becomes the intercept and then isolate if significantly different from that. Does this mean Origin has a significant effect on the value?
Would the better approach be to do the following?:
mcomp = glm.nb(value ~ origin + Substrate, data = comb_data) 
summary(aov(mcomp))
              Df Sum Sq Mean Sq F value Pr(>F)    
origin         1     23   22.55   6.612 0.0102 *  
Substrate     44   1445   32.84   9.631 <2e-16 ***
Residuals   2634   8981    3.41                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 

This shows me that origin and substrate have an effect on value if I understand correctly?

**Tags:** modelling, linear-regression, statsmodels

---

## 2 answers

### Answer by username_1 on 2020-06-27. Score: 1

Second viewing of the question from what I can see -0.22 as a coefficient of origin is a strong negative association, so yeah it has a major impact. Its not how I would have done it, but that looks to be the result.

---

First viewing,

I'm going to throw my hat in here. We don't know what 'origin' is about, anyway just throw everything, i.e. each substrate and the origin into the same regression calculation. Check for a low-residual and preferably do a Q-Q plot, transform your data it this doesn't look good.

The key and the thing you are missing is your regression weights, without that I couldn't say very much. If the regression weight is near zero for 'origin' then it has zero impact. If the regression weight of 'origin' is positively greater than everything else ... I assume there are skewed distributions of 'substrates' between the 'origins'. If the regression weight of 'origin' is negative but still greater than all other regression weights then it is adversely affecting the 'value' you are seeking.

I don't know the experiment, the biological system or really the 'substrate' assays, so I can't comment any further.

The two issues I have are:

1. Doing an ANOVA on the output of a regression analysis doesn't make much sense to me. It is not something I would do, nor something in ML or GLM I've encountered.
2. Are you doing pairwise substrate/origin calculations? I presue not, but just in case this not how GLM works.

> **Comments:**
> **username_0**: oh I was under the impression `aov(summary(glm_model))` was a pretty common practice as it nicely summarises down to a variable level? Where do I find the regression weights?  
> **username_0**: When transforming the data do I have to transform both variables or can I transform just one of them until they fit the same distribution? and given that I am using a negative binomial glm does a uniform distribution between the variables matter as much?  I am unsure if there is a skewed distribution of variables between origins, that is what I am looking to find out really.  
> **username_2**: hi @Michael, very nice points here, yes the term anova is quite misleading sometimes. Normally people associate associate with the classical analysis of variance. In more "modern" uses, it can refer to the breakdown of deviance in a model, https://stats.stackexchange.com/questions/289226/what-are-the-differences-between-anovas-and-glms  
> **username_2**: It is like a log likelihood ratio test essentially, the "F" statistic instead of Chi-sq is used.  
> **username_0**: @StupidWolf So it is fine to `aov(summary(glm_model))` to get a nice overview of the model? And by pairwise substrate/origin calculations does you mean doing the model on a subsatrate of the dataframe for each substrate @michael?  

---

### Answer by username_2 on 2020-06-27. Score: 0

There is no better method, it's a matter of what you want to test or what is your question.

Using the `anova()` or `aov()`, test the terms collectively. For example, in your example with Substrate, the null hypothesis is that the coefficients are all zero, meaning `cellulose =0, mannan =0 , ....`

If the question is, "do the isolate samples have a higher value than origin samples?", then you can use your first model, where `free` is set as the reference and you test whether the effect of `isolate` is non-zero. Likewise you can do this for substrate and set of them as your reference. You can also do other pairwise comparisons using this model.

If the question is, "does origin have a significant effect on value, after controlling for substrate?", then you can use your second model.

> **Comments:**
> **username_0**: Would I control for substrate by forming the model as such: value ~ origin + substrate ?  
> **username_2**: yes you do. you are giving every substrate a meaning  and on top of that estimating the effect of origin  
> **username_0**: And does the order in which I give the variable to the model, i.e. origin before substrate, indicate some sort of predefined importance to the model?  
> **username_2**: it doesn't define the importance, you can put it in any order. However if they are correlated, most of the variance will go to the first. I don't think this will apply in your case  
> **username_2**: see https://cran.r-project.org/doc/contrib/Faraway-PRA.pdf blocking design. A good ieda to read this  
> **username_0**: Thank you I will look at this :)  

---

