  Homomorphic encryption (HE) is a promising cryptographic technique for
enabling secure collaborative machine learning in the cloud. However, support
for homomorphic computation on ciphertexts under multiple keys is inefficient.
Current solutions often require key setup before any computation or incur large
ciphertext size (at best, grow linearly to the number of involved keys). In
this paper, we proposed a new approach that leverages threshold and multi-key
HE to support computations on ciphertexts under different keys. Our new
approach removes the need for key setup between each client and the set of
model owners. At the same time, this approach reduces the number of encrypted
models to be offloaded to the cloud evaluator, and the ciphertext size with a
dimension reduction from (N+1)x2 to 2x2. We present the details of each step
and discuss the complexity and security of our approach.
