  Questionnaires in the behavioral and organizational sciences tend to be
lengthy: survey measures comprising hundreds of items are the norm rather than
the exception. However, literature suggests that the longer a questionnaire
takes, the higher the probability that participants lose interest and start
responding carelessly. Consequently, in long surveys a large number of
participants may engage in careless responding, posing a major threat to
internal validity. We propose a novel method for identifying the onset of
careless responding (or an absence thereof) for each participant. It is based
on combined measurements of multiple dimensions in which carelessness may
manifest, such as inconsistency and invariability. Since a structural break in
either dimension is potentially indicative of carelessness, the proposed method
searches for evidence for changepoints along the combined measurements. It is
highly flexible, based on machine learning, and provides statistical guarantees
on its performance. An empirical application on data from a seminal study on
the incidence of careless responding reveals that the reported incidence has
likely been substantially underestimated due to the presence of respondents
that were careless for only parts of the questionnaire. In simulation
experiments, we find that the proposed method achieves high reliability in
correctly identifying carelessness onset, discriminates well between careless
and attentive respondents, and captures a variety of careless response types,
even when a large number of careless respondents are present. Furthermore, we
provide freely available open source software to enhance accessibility and
facilitate adoption by empirical researchers.
