  Rapid growth in scientific data and a widening gap between computational
speed and I/O bandwidth make it increasingly infeasible to store and share all
data produced by scientific simulations. Instead, we need methods for reducing
data volumes: ideally, methods that can scale data volumes adaptively so as to
enable negotiation of performance and fidelity tradeoffs in different
situations. Multigrid-based hierarchical data representations hold promise as a
solution to this problem, allowing for flexible conversion between different
fidelities so that, for example, data can be created at high fidelity and then
transferred or stored at lower fidelity via logically simple and mathematically
sound operations. However, the effective use of such representations has been
hindered until now by the relatively high costs of creating, accessing,
reducing, and otherwise operating on such representations. We describe here
highly optimized data refactoring kernels for GPU accelerators that enable
efficient creation and manipulation of data in multigrid-based hierarchical
forms. We demonstrate that our optimized design can achieve up to 250 TB/s
aggregated data refactoring throughput -- 83% of theoretical peak -- on 1024
nodes of the Summit supercomputer. We showcase our optimized design by applying
it to a large-scale scientific visualization workflow and the MGARD lossy
compression software.
