  Hierarchical model fitting has become commonplace for case-control studies of
cognition and behaviour in mental health. However, these techniques require us
to formalise assumptions about the data-generating process at the group level,
which may not be known. Specifically, researchers typically must choose whether
to assume all subjects are drawn from a common population, or to model them as
deriving from separate populations. These assumptions have profound
implications for computational psychiatry, as they affect the resulting
inference (latent parameter recovery) and may conflate or mask true group-level
differences. To test these assumptions we ran systematic simulations on
synthetic multi-group behavioural data from a commonly used multi-armed bandit
task (reinforcement learning task). We then examined recovery of group
differences in latent parameter space under the two commonly used generative
modelling assumptions: (1) modelling groups under a common shared group-level
prior (assuming all participants are generated from a common distribution, and
are likely to share common characteristics); (2) modelling separate groups
based on symptomatology or diagnostic labels, resulting in separate group-level
priors. We evaluated the robustness of these approaches to variations in data
quality and prior specifications on a variety of metrics. We found that fitting
groups separately (assumptions 2), provided the most accurate and robust
inference across all conditions. Our results suggest that when dealing with
data from multiple clinical groups, researchers should analyse patient and
control groups separately as it provides the most accurate and robust recovery
of the parameters of interest.
