  A common approach when studying the quality of representation involves
comparing the latent preferences of voters and legislators, commonly obtained
by fitting an item-response theory (IRT) model to a common set of stimuli.
Despite being exposed to the same stimuli, voters and legislators may not share
a common understanding of how these stimuli map onto their latent preferences,
leading to differential item-functioning (DIF) and incomparability of
estimates. We explore the presence of DIF and incomparability of latent
preferences obtained through IRT models by re-analyzing an influential survey
data set, where survey respondents expressed their preferences on roll call
votes that U.S. legislators had previously voted on. To do so, we propose
defining a Dirichlet Process prior over item-response functions in standard IRT
models. In contrast to typical multi-step approaches to detecting DIF, our
strategy allows researchers to fit a single model, automatically identifying
incomparable sub-groups with different mappings from latent traits onto
observed responses. We find that although there is a group of voters whose
estimated positions can be safely compared to those of legislators, a sizeable
share of surveyed voters understand stimuli in fundamentally different ways.
Ignoring these issues can lead to incorrect conclusions about the quality of
representation.
