  A growing body of literature has focused on detailing the linguistic
knowledge embedded in large, pretrained language models. Existing work has
shown that non-linguistic biases in models can drive model behavior away from
linguistic generalizations. We hypothesized that competing linguistic processes
within a language, rather than just non-linguistic model biases, could obscure
underlying linguistic knowledge. We tested this claim by exploring a single
phenomenon in four languages: English, Chinese, Spanish, and Italian. While
human behavior has been found to be similar across languages, we find
cross-linguistic variation in model behavior. We show that competing processes
in a language act as constraints on model behavior and demonstrate that
targeted fine-tuning can re-weight the learned constraints, uncovering
otherwise dormant linguistic knowledge in models. Our results suggest that
models need to learn both the linguistic constraints in a language and their
relative ranking, with mismatches in either producing non-human-like behavior.
