  Derivationally related words, such as "runner" and "running", exhibit
semantic differences which also elicit different visual scenarios. In this
paper, we ask whether Vision and Language (V\&L) models capture such
distinctions at the morphological level, using a a new methodology and dataset.
We compare the results from V\&L models to human judgements and find that
models' predictions differ from those of human participants, in particular
displaying a grammatical bias. We further investigate whether the human-model
misalignment is related to model architecture. Our methodology, developed on
one specific morphological contrast, can be further extended for testing models
on capturing other nuanced language features.
