  Real-time computation of data streams over affordable virtualized
infrastructure resources is an important form of data in motion processing
architecture. However, processing such data streams while ensuring strict
guarantees on quality of services is problematic due to: (i) uncertain stream
arrival pattern; (ii) need of processing different types of continuous queries;
and (iii) variable resource consumption behavior of continuous queries. Recent
work has explored the use of statistical techniques for resource estimation of
SQL queries and OLTP workloads. All these techniques approximate resource usage
for each query as a single point value. However, in data stream processing
workloads in which data flows through the graph of operators endlessly and
poses performance and resource demand fluctuations, the single point resource
estimation is inadequate. Because it is neither expressive enough nor does it
capture the multi-modal nature of the target data. To this end, we present a
novel technique which uses mixture density networks, a combined structure of
neural networks and mixture models, to estimate the whole spectrum of resource
usage as probability density functions. The proposed approach is a flexible and
convenient means of modeling unknown distribution models. We have validated the
models using both the linear road benchmark and the TPC-H, observing high
accuracy under a number of error metrics: mean-square error, continuous ranked
probability score, and negative log predictive density.
