  A cloud-based data stream management system (DSMS) handles fast data by
utilizing the massively parallel processing capabilities of the underlying
platform. An important property of such a DSMS is elasticity, meaning that
nodes can be dynamically added to or removed from an application to match the
latter's workload, which may fluctuate in an unpredictable manner. For an
application involving stateful operations such as aggregates, the addition /
removal of nodes necessitates the migration of operator states. Although the
importance of migration has been recognized in existing systems, two key
problems remain largely neglected, namely how to migrate and what to migrate,
i.e., the migration mechanism that reduces synchronization overhead and result
delay during migration, and the selection of the optimal task assignment that
minimizes migration costs. Consequently, migration in current systems typically
incurs a high spike in result delay caused by expensive synchronization
barriers and suboptimal task assignments. Motivated by this, we present the
first comprehensive study on efficient operator states migration, and propose
designs and algorithms that enable live, progressive, and optimized migrations.
Extensive experiments using real data justify our performance claims.
