  In the era of data explosion, a growing number of data-intensive computing
frameworks, such as Apache Hadoop and Spark, have been proposed to handle the
massive volume of unstructured data in parallel. Since programming models
provided by these frameworks allow users to specify complex and diversified
user-defined functions (UDFs) with predefined operations, the grand challenge
of tuning up entire system performance arises if programmers do not fully
understand the semantics of code, data, and runtime systems. In this paper, we
design a holistic semantics-aware optimization for data-intensive applications
using hybrid program analysis} (SODA) to assist programmers to tune performance
issues. SODA is a two-phase framework: the offline phase is a static analysis
that analyzes code and performance profiling data from the online phase of
prior executions to generate a parameterized and instrumented application; the
online phase is a dynamic analysis that keeps track of the application's
execution and collects runtime information of data and system. Extensive
experimental results on four real-world Spark applications show that SODA can
gain up to 60%, 10%, 8%, faster than its original implementation, with the
three proposed optimization strategies, i.e., cache management, operation
reordering, and element pruning, respectively.
