  Today, data analysis drives the decision-making process in virtually every
human activity. This demands for software platforms that offer simple
programming abstractions to express data analysis tasks and that can execute
them in an efficient and scalable way. State-of-the-art solutions range from
low-level programming primitives, which give control to the developer about
communication and resource usage, but require significant effort to develop and
optimize new algorithms, to high-level platforms that hide most of the
complexities of parallel and distributed processing, but often at the cost of
reduced efficiency. To reconcile these requirements, we developed Renoir, a
novel distributed data processing platform written in Rust. Renoir provides a
high-level dataflow programming model as mainstream data processing systems. It
supports static and streaming data, it enables data transformations, grouping,
aggregation, iterative computations, and time-based analytics, incurring in a
low overhead. This paper presents In this paper, we present the programming
model and the implementation details of Renoir. We evaluate it under
heterogeneous workloads. We compare it with state-of-the-art solutions for data
analysis and high-performance computing, as well as alternative research
products, which offer different programming abstractions and implementation
strategies. Renoir programs are compact and easy to write: developers need not
care about low-level concerns such as resource usage, data serialization,
concurrency control, and communication. Renoir consistently presents comparable
or better performance than competing solutions, by a large margin in several
scenarios. We conclude that Renoir offers a good tradeoff between simplicity
and performance, allowing developers to easily express complex data analysis
tasks and achieve high performance and scalability.
