  The era of big data has led to the emergence of new systems for real-time
distributed stream processing, e.g., Apache Storm is one of the most popular
stream processing systems in industry today. However, Storm, like many other
stream processing systems lacks an intelligent scheduling mechanism. The
default round-robin scheduling currently deployed in Storm disregards resource
demands and availability, and can therefore be inefficient at times. We present
R-Storm (Resource-Aware Storm), a system that implements resource-aware
scheduling within Storm. R-Storm is designed to increase overall throughput by
maximizing resource utilization while minimizing network latency. When
scheduling tasks, R-Storm can satisfy both soft and hard resource constraints
as well as minimizing network distance between components that communicate with
each other. We evaluate R-Storm on set of micro-benchmark Storm applications as
well as Storm applications used in production at Yahoo! Inc. From our
experimental results we conclude that R-Storm achieves 30-47% higher throughput
and 69-350% better CPU utilization than default Storm for the micro-benchmarks.
For the Yahoo! Storm applications, R-Storm outperforms default Storm by around
50% based on overall throughput. We also demonstrate that R-Storm performs much
better when scheduling multiple Storm applications than default Storm.
