  Recently, large multimodal models, such as CLIP and Stable Diffusion have
experimented tremendous successes in both foundations and applications.
However, as these models increase in parameter size and computational
requirements, it becomes more challenging for users to personalize them for
specific tasks or preferences. In this work, we address the problem of adapting
the previous models towards sets of particular human preferences, aligning the
retrieved or generated images with the preferences of the user. We leverage the
Bradley-Terry preference model to develop a fast adaptation method that
efficiently fine-tunes the original model, with few examples and with minimal
computing resources. Extensive evidence of the capabilities of this framework
is provided through experiments in different domains related to multimodal text
and image understanding, including preference prediction as a reward model, and
generation tasks.
