  Checkpointing (C) and restoring (R) are key components for GPU tasks. POS is
an OS-level GPU C/R system: It can transparently checkpoint or restore
processes that use the GPU, without requiring any cooperation from the
application, a key feature required by modern systems like the cloud. Moreover,
POS is the first OS-level C/R system that can concurrently execute C/R with the
application execution: a critical feature that can be trivially achieved when
the processes only running on the CPU, but becomes challenging when the
processes use GPU. The problem is how to ensure consistency during concurrent
execution with the lack of application semantics due to transparency. CPU
processes can leverage OS and hardware paging to fix inconsistency without
application semantics. Unfortunately, GPU bypasses OS and paging for high
performance. POS fills the semantic gap by speculatively extracting buffer
access information of GPU kernels during runtime. Thanks to the simple and
well-structured nature of GPU kernels, our speculative extraction (with runtime
validation) achieves 100% accuracy on applications from training to inference
whose domains span from vision, large language models, and reinforcement
learning. Based on the extracted semantics, we systematically overlap C/R with
application execution, and achieves orders of magnitude higher performance
under various tasks compared with the state-of-the-art OS-level GPU C/R,
including training fault tolerance, live GPU process migration, and cold starts
acceleration in GPU-based serverless computing.
