  Maintaining the integrity of long-term data collection is an essential
scientific practice. As a field evolves, so too will that field's measurement
instruments and data storage systems, as they are invented, improved upon, and
made obsolete. For data streams generated by opaque sociotechnical systems
which may have episodic and unknown internal rule changes, detecting and
accounting for shifts in historical datasets requires vigilance and creative
analysis. Here, we show that around 10\% of day-scale word usage frequency time
series for Twitter collected in real time for a set of roughly 10,000
frequently used words for over 10 years come from tweets with, in effect,
corrupted language labels. We describe how we uncovered problematic signals
while comparing word usage over varying time frames. We locate time points
where Twitter switched on or off different kinds of language identification
algorithms, and where data formats may have changed. We then show how we create
a statistic for identifying and removing words with pathological time series.
While our resulting process for removing `bad' time series from ensembles of
time series is particular, the approach leading to its construction may be
generalizeable.
