  MLaaS (Machine Learning as a Service) has become popular in the cloud
computing domain, allowing users to leverage cloud resources for running
private inference of ML models on their data. However, ensuring user input
privacy and secure inference execution is essential. One of the approaches to
protect data privacy and integrity is to use Trusted Execution Environments
(TEEs) by enabling execution of programs in secure hardware enclave. Using TEEs
can introduce significant performance overhead due to the additional layers of
encryption, decryption, security and integrity checks. This can lead to slower
inference times compared to running on unprotected hardware. In our work, we
enhance the runtime performance of ML models by introducing layer partitioning
technique and offloading computations to GPU. The technique comprises two
distinct partitions: one executed within the TEE, and the other carried out
using a GPU accelerator. Layer partitioning exposes intermediate feature maps
in the clear which can lead to reconstruction attacks to recover the input. We
conduct experiments to demonstrate the effectiveness of our approach in
protecting against input reconstruction attacks developed using trained
conditional Generative Adversarial Network(c-GAN). The evaluation is performed
on widely used models such as VGG-16, ResNet-50, and EfficientNetB0, using two
datasets: ImageNet for Image classification and TON IoT dataset for
cybersecurity attack detection.
