  Applications ranging from algorithmic trading to scientific data analysis
require realtime analytics based on views over databases that change at very
high rates. Such views have to be kept fresh at low maintenance cost and
latencies. At the same time, these views have to support classical SQL, rather
than window semantics, to enable applications that combine current with aged or
historical data. In this paper, we present viewlet transforms, a recursive
finite differencing technique applied to queries. The viewlet transform
materializes a query and a set of its higher-order deltas as views. These views
support each other's incremental maintenance, leading to a reduced overall view
maintenance cost. The viewlet transform of a query admits efficient evaluation,
the elimination of certain expensive query operations, and aggressive
parallelization. We develop viewlet transforms into a workable query execution
technique, present a heuristic and cost-based optimization framework, and
report on experiments with a prototype dynamic data management system that
combines viewlet transforms with an optimizing compilation technique. The
system supports tens of thousands of complete view refreshes a second for a
wide range of queries.
