  To process a large volume of data, modern data management systems use a
collection of machines connected through a network. This paper looks into the
feasibility of scaling up such a shared-nothing system while processing a
compute- and communication-intensive workload---processing distributed joins.
By exploiting multiple processing cores within the individual machines, we
implement a system to process database joins that parallelizes computation
within each node, pipelines the computation with communication, parallelizes
the communication by allowing multiple simultaneous data transfers
(send/receive), and removes synchronization barriers (a scalability bottleneck
in a distributed data processing system). Our experimental results show that
using only four threads per node the framework achieves a 3.5x gains in
intra-node performance while compared with a single-threaded counterpart.
Moreover, with the join processing workload the cluster-wide performance (and
speedup) is observed to be dictated by the intra-node computational loads; this
property brings a near-linear speedup with increasing nodes in the system, a
feature much desired in modern large-scale data processing system.
