  Object-centric (OC) representations, which represent the state of a visual
scene by modeling it as a composition of objects, have the potential to be used
in various downstream tasks to achieve systematic compositional generalization
and facilitate reasoning. However, these claims have not been thoroughly
analyzed yet. Recently, foundation models have demonstrated unparalleled
capabilities across diverse domains from language to computer vision, marking
them as a potential cornerstone of future research for a multitude of
computational tasks. In this paper, we conduct an extensive empirical study on
representation learning for downstream Visual Question Answering (VQA), which
requires an accurate compositional understanding of the scene. We thoroughly
investigate the benefits and trade-offs of OC models and alternative approaches
including large pre-trained foundation models on both synthetic and real-world
data, and demonstrate a viable way to achieve the best of both worlds. The
extensiveness of our study, encompassing over 800 downstream VQA models and 15
different types of upstream representations, also provides several additional
insights that we believe will be of interest to the community at large.
