  Tabular representation learning has recently gained a lot of attention.
However, existing approaches only learn a representation from a single table,
and thus ignore the potential to learn from the full structure of relational
databases, including neighboring tables that can contain important information
for a contextualized representation. Moreover, current models are significantly
limited in scale, which prevents that they learn from large databases. In this
paper, we thus introduce our vision of relational representation learning, that
can not only learn from the full relational structure, but also can scale to
larger database sizes that are commonly found in real-world. Moreover, we also
discuss opportunities and challenges we see along the way to enable this vision
and present initial very promising results. Overall, we argue that this
direction can lead to foundation models for relational databases that are today
only available for text and images.
