  The data science community today has embraced the concept of Dataframes as
the de facto standard for data representation and manipulation. Ease of use,
massive operator coverage, and popularization of R and Python languages have
heavily influenced this transformation. However, most widely used serial
Dataframes today (R, pandas) experience performance limitations even while
working on even moderately large data sets. We believe that there is plenty of
room for improvement by investigating the generic distributed patterns of
dataframe operators. In this paper, we propose a framework that lays the
foundation for building high performance distributed-memory parallel dataframe
systems based on these parallel processing patterns. We also present Cylon, as
a reference runtime implementation. We demonstrate how this framework has
enabled Cylon achieving scalable high performance. We also underline the
flexibility of the proposed API and the extensibility of the framework on
different hardware. To the best of our knowledge, Cylon is the first and only
distributed-memory parallel dataframe system available today.
