  A computing job in a big data system can take a long time to run, especially
for pipelined executions on data streams. Developers often need to change the
computing logic of the job such as fixing a loophole in an operator or changing
the machine learning model in an operator with a cheaper model to handle a
sudden increase of the data-ingestion rate. Recently many systems have started
supporting runtime reconfigurations to allow this type of change on the fly
without killing and restarting the execution. While the delay in
reconfiguration is critical to performance, existing systems use epochs to do
runtime reconfigurations, which can cause a long delay. In this paper we
develop a new technique called Fries that leverages the emerging availability
of fast control messages in many systems, since these messages can be sent
without being blocked by data messages. We formally define consistency in
runtime reconfigurations, and develop a Fries scheduler with consistency
guarantees. The technique not only works for different classes of dataflows,
but also works for parallel executions and supports fault tolerance. Our
extensive experimental evaluation on clusters show the advantages of this
technique compared to epoch-based schedulers.
