  We study the problem of discovering joinable datasets at scale. This is, how
to automatically discover pairs of attributes in a massive collection of
independent, heterogeneous datasets that can be joined. Exact (e.g., based on
distinct values) and hash-based (e.g., based on locality-sensitive hashing)
techniques require indexing the entire dataset, which is unattainable at scale.
To overcome this issue, we approach the problem from a learning perspective
relying on profiles. These are succinct representations that capture the
underlying characteristics of the schemata and data values of datasets, which
can be efficiently extracted in a distributed and parallel fashion. Profiles
are then compared, to predict the quality of a join operation among a pair of
attributes from different datasets. In contrast to the state-of-the-art, we
define a novel notion of join quality that relies on a metric considering both
the containment and cardinality proportions between candidate attributes. We
implement our approach in a system called NextiaJD, and present extensive
experiments to show the predictive performance and computational efficiency of
our method. Our experiments show that NextiaJD obtains similar predictive
performance to that of hash-based methods, yet we are able to scale-up to
larger volumes of data. Also, NextiaJD generates a considerably less amount of
false positives, which is a desirable feature at scale.
