  Human languages are rule governed, but almost invariably these rules have
exceptions in the form of irregularities. Since rules in language are efficient
and productive, the persistence of irregularity is an anomaly. How does
irregularity linger in the face of internal (endogenous) and external
(exogenous) pressures to conform to a rule? Here we address this problem by
taking a detailed look at simple past tense verbs in the Corpus of Historical
American English. The data show that the language is open, with many new verbs
entering. At the same time, existing verbs might tend to regularize or
irregularize as a consequence of internal dynamics, but overall, the amount of
irregularity sustained by the language stays roughly constant over time.
Despite continuous vocabulary growth, and presumably, an attendant increase in
expressive power, there is no corresponding growth in irregularity. We analyze
the set of irregulars, showing they may adhere to a set of minority rules,
allowing for increased stability of irregularity over time. These findings
contribute to the debate on how language systems become rule governed, and how
and why they sustain exceptions to rules, providing insight into the interplay
between the emergence and maintenance of rules and exceptions in language.
