  In this paper, we focus on general-purpose Distributed Stream Data Processing
Systems (DSDPSs), which deal with processing of unbounded streams of continuous
data at scale distributedly in real or near-real time. A fundamental problem in
a DSDPS is the scheduling problem with the objective of minimizing average
end-to-end tuple processing time. A widely-used solution is to distribute
workload evenly over machines in the cluster in a round-robin manner, which is
obviously not efficient due to lack of consideration for communication delay.
Model-based approaches do not work well either due to the high complexity of
the system environment. We aim to develop a novel model-free approach that can
learn to well control a DSDPS from its experience rather than accurate and
mathematically solvable system models, just as a human learns a skill (such as
cooking, driving, swimming, etc). Specifically, we, for the first time, propose
to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free
control in DSDPSs; and present design, implementation and evaluation of a novel
and highly effective DRL-based control framework, which minimizes average
end-to-end tuple processing time by jointly learning the system environment via
collecting very limited runtime statistics data and making decisions under the
guidance of powerful Deep Neural Networks. To validate and evaluate the
proposed framework, we implemented it based on a widely-used DSDPS, Apache
Storm, and tested it with three representative applications. Extensive
experimental results show 1) Compared to Storm's default scheduler and the
state-of-the-art model-based method, the proposed framework reduces average
tuple processing by 33.5% and 14.0% respectively on average. 2) The proposed
framework can quickly reach a good scheduling solution during online learning,
which justifies its practicability for online control in DSDPSs.
