  Modern GPU applications, such as machine learning (ML) frameworks, can only
partially utilize beefy GPUs, leading to GPU underutilization in cloud
environments. Sharing GPUs across multiple applications from different users
can improve resource utilization and consequently cost, energy, and power
efficiency. However, GPU sharing creates memory safety concerns because kernels
must share a single GPU address space (GPU context). Previous GPU memory
protection approaches have limited deployability because they require
specialized hardware extensions or access to source code. This is often
unavailable in GPU-accelerated libraries heavily utilized by ML frameworks. In
this paper, we present G-Safe, a PTX-level bounds checking approach for GPUs
that limits GPU kernels of each application to stay within the memory partition
allocated to them. G-Safe relies on three mechanisms: (1) It divides the common
GPU address space into separate partitions for different applications. (2) It
intercepts and checks data transfers, fencing erroneous operations. (3) It
instruments all GPU kernels at the PTX level (available in closed GPU
libraries) fencing all kernel memory accesses outside application memory
bounds. We implement G-Safe as an external, dynamically linked library that can
be pre-loaded at application startup time. G-Safe's approach is transparent to
applications and can support real-life, complex frameworks, such as Caffe and
PyTorch, that issue billions of GPU kernels. Our evaluation shows that the
overhead of G-Safe compared to native (unprotected) for such frameworks is
between 4\% - 12\% and on average 9\%.
