<p>As an update, this paper may be helpful, though it comes from the medical field.</p>
<p><strong>References</strong></p>
<p><a href="https://rdcu.be/cyKnT" rel="nofollow noreferrer">Monsalves, M.J., Bangdiwala, A.S., Thabane, A. et al. LEVEL (Logical Explanations &amp; Visualizations of Estimates in Linear mixed models): recommendations for reporting multilevel data and analyses. BMC Med Res Methodol 20, 3 (2020).</a></p>

CC BY-SA 4.0
2021-10-03T07:48:28.250Z
2021-10-03T07:48:28.250Z
NewMoonPalace
<p>There are some informative examples in Baayen, Davidson and Bates (2008), though some of their advice is outdated, having been supplanted by Barr et al. (2013) cited in the answer above. I found it useful to read these two papers together, though.</p>
<p>I'd like to add my voice to @Christian in stressing that one common gap in reporting such models is which random effects included just an intercept, and which (if any) included an intercept and a slope, along with a justification of of these decisions. Baayen et al. provide some examples of such justifications.</p>
<h3>References</h3>
<p>Baayen, R. H, Davidson, D. J., Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.</p>

CC BY-SA 3.0
2015-05-15T17:48:33.683Z
2015-05-15T18:08:56.230Z
2020-06-17T09:55:31.633Z
Justin
<p>The APA style manual does not provide specific guidelines for linear mixed models. Additionally, a review of studies using linear mixed models reported that the psychological papers surveyed differed 'substantially' in how they reported on these models (Barr, Levy, Scheepers and Tily, 2013). It depends greatly on your study, in other words. Normatively speaking, I find <a href="http://www.infoagepub.com/products/content/files/serlinfiles_multilevel/Chapter%2011/checklist.pdf" rel="noreferrer">this checklist</a> (Ferron et al., 2008) extremely helpful in such cases.</p>
<p>Besides the more general checklist, Barr, Levy, Scheepers and Tily (2013) also provide some guidelines for reporting on LMMs. The whole paper is well worth a read, but excerpts of their &quot;Reporting results&quot; section are provided below.</p>
<blockquote>
<p>One needs to provide sufficient information for the reader to be able to recreate the analyses. One way of satisfying this requirement is to <strong>report the variance–covariance matrix</strong>, which includes all the information about the random effects, including their estimates. ... A simpler option is to mention that one attempted to use a maximal LMEM and, as an added check, also state which factors had random slopes associated with them.</p>
<p>If it is seen as necessary or desirable in a confirmatory analysis to determine the random effects structure using a data-driven approach, certain minimal guidelines should be followed. First, it is critical to <strong>report the criteria that have been used, including the a-level for exclusion/inclusion of
random slopes and the order in which random slopes were tested.</strong> Furthermore, authors should explicitly report the changed assumptions about the generative process underlying the data that result from excluding the random slope (rather than just stating that the slopes did not ‘‘improve model fit’’), and should do so in non-technical language that non-experts can understand. Readers with only background in ANOVA will not understand that removing the random slope corresponds to pooling error across strata in a mixed-model ANOVA analysis. It is therefore preferable to <strong>clearly state the underlying assumption of a constant effect</strong>, e.g., ‘‘by excluding the random slope for the priming manipulation, we assume that the priming effect is invariant across subjects (or items) in the population.’’</p>
</blockquote>
<p>Searching the Barr article, I also found a highly relevant <a href="https://stats.stackexchange.com/q/26855/23654">CrossVal.SE</a> question which asks for examples rather than guidelines. Since no guidelines appear to exist, I would suggest that.</p>
<h3>References</h3>
<ul>
<li><a href="http://idiom.ucsd.edu/%7Erlevy/papers/barr-etal-2013-jml.pdf" rel="noreferrer">Barr, D. J., Levy, R., Scheepers, C., &amp; Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. doi:10.1016/j.jml.2012.11.001</a></li>
<li><a href="http://www.infoagepub.com/products/content/files/serlinfiles_multilevel/Chapter%2011/checklist.pdf" rel="noreferrer">Ferron, J. M., Hogarty, K. Y., Dedrick, R. F., Hess, M. R., Niles, J. D., &amp; Kromrey, J. D. (2008). Reporting results from multilevel analyses. Multilevel modeling of educational data, 391-426.</a> (Statistical checklist only.)</li>
</ul>

CC BY-SA 4.0
2024-09-12T12:11:54.917Z
New link for the Barr 2013 paper, which was 404 for me today: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3881361/pdf/nihms533954.pdf
Chris Andrews
CC BY-SA 3.0
2015-05-15T08:47:03.353Z
2015-05-15T08:47:03.353Z
2020-06-17T09:55:31.633Z
Christian Hummeluhr
<p>Are there any official APA guidelines for reporting linear mixed model results?</p>

<p>Can I take an approach similar to what would be done with an ANOVA (i.e., report the p-value, standard error and .95-CI for each main effect and interaction)?</p>

<p>What about the intercept?  Any additional recommendations?</p>

<statistics><linear-mixed-model><regression><publication><apa-style>
How should results from linear mixed models (LMMs) be reported?
Louis Thibault
# How should results from linear mixed models (LMMs) be reported?

(Asked by: username_0 on 2015-04-17)

Are there any official APA guidelines for reporting linear mixed model results?

Can I take an approach similar to what would be done with an ANOVA (i.e., report the p-value, standard error and .95-CI for each main effect and interaction)?

What about the intercept? Any additional recommendations?

**Tags:** statistics, linear-mixed-model, regression, publication, apa-style


---

## 3 answers

### Answer by username_3 on 2015-05-15. Score: 7 | Accepted answer

The APA style manual does not provide specific guidelines for linear mixed models. Additionally, a review of studies using linear mixed models reported that the psychological papers surveyed differed 'substantially' in how they reported on these models (Barr, Levy, Scheepers and Tily, 2013). It depends greatly on your study, in other words. Normatively speaking, I find [this checklist](http://www.infoagepub.com/products/content/files/serlinfiles_multilevel/Chapter%2011/checklist.pdf) (Ferron et al., 2008) extremely helpful in such cases.

Besides the more general checklist, Barr, Levy, Scheepers and Tily (2013) also provide some guidelines for reporting on LMMs. The whole paper is well worth a read, but excerpts of their "Reporting results" section are provided below.

> One needs to provide sufficient information for the reader to be able to recreate the analyses. One way of satisfying this requirement is to **report the variance–covariance matrix**, which includes all the information about the random effects, including their estimates. ... A simpler option is to mention that one attempted to use a maximal LMEM and, as an added check, also state which factors had random slopes associated with them.
>
> If it is seen as necessary or desirable in a confirmatory analysis to determine the random effects structure using a data-driven approach, certain minimal guidelines should be followed. First, it is critical to **report the criteria that have been used, including the a-level for exclusion/inclusion of
> random slopes and the order in which random slopes were tested.** Furthermore, authors should explicitly report the changed assumptions about the generative process underlying the data that result from excluding the random slope (rather than just stating that the slopes did not ‘‘improve model fit’’), and should do so in non-technical language that non-experts can understand. Readers with only background in ANOVA will not understand that removing the random slope corresponds to pooling error across strata in a mixed-model ANOVA analysis. It is therefore preferable to **clearly state the underlying assumption of a constant effect**, e.g., ‘‘by excluding the random slope for the priming manipulation, we assume that the priming effect is invariant across subjects (or items) in the population.’’

Searching the Barr article, I also found a highly relevant [CrossVal.SE](https://stats.stackexchange.com/q/26855/23654) question which asks for examples rather than guidelines. Since no guidelines appear to exist, I would suggest that.

### References

* [Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. doi:10.1016/j.jml.2012.11.001](http://idiom.ucsd.edu/%7Erlevy/papers/barr-etal-2013-jml.pdf)
* [Ferron, J. M., Hogarty, K. Y., Dedrick, R. F., Hess, M. R., Niles, J. D., & Kromrey, J. D. (2008). Reporting results from multilevel analyses. Multilevel modeling of educational data, 391-426.](http://www.infoagepub.com/products/content/files/serlinfiles_multilevel/Chapter%2011/checklist.pdf) (Statistical checklist only.)

> **Comments:**
> **username_4**: New link for the Barr 2013 paper, which was 404 for me today: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3881361/pdf/nihms533954.pdf  

---

### Answer by username_1 on 2021-10-03. Score: 3

As an update, this paper may be helpful, though it comes from the medical field.

**References**

[Monsalves, M.J., Bangdiwala, A.S., Thabane, A. et al. LEVEL (Logical Explanations & Visualizations of Estimates in Linear mixed models): recommendations for reporting multilevel data and analyses. BMC Med Res Methodol 20, 3 (2020).](https://rdcu.be/cyKnT)


---

### Answer by username_2 on 2015-05-15. Score: 2

There are some informative examples in Baayen, Davidson and Bates (2008), though some of their advice is outdated, having been supplanted by Barr et al. (2013) cited in the answer above. I found it useful to read these two papers together, though.

I'd like to add my voice to @Christian in stressing that one common gap in reporting such models is which random effects included just an intercept, and which (if any) included an intercept and a slope, along with a justification of of these decisions. Baayen et al. provide some examples of such justifications.

### References

Baayen, R. H, Davidson, D. J., Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.


---

