  Federated learning (FL) allows multiple participants to collaboratively build
deep learning (DL) models without directly sharing data. Consequently, the
issue of copyright protection in FL becomes important since unreliable
participants may gain access to the jointly trained model. Application of
homomorphic encryption (HE) in secure FL framework prevents the central server
from accessing plaintext models. Thus, it is no longer feasible to embed the
watermark at the central server using existing watermarking schemes. In this
paper, we propose a novel client-side FL watermarking scheme to tackle the
copyright protection issue in secure FL with HE. To our best knowledge, it is
the first scheme to embed the watermark to models under the Secure FL
environment. We design a black-box watermarking scheme based on client-side
backdooring to embed a pre-designed trigger set into an FL model by a
gradient-enhanced embedding method. Additionally, we propose a trigger set
construction mechanism to ensure the watermark cannot be forged. Experimental
results demonstrate that our proposed scheme delivers outstanding protection
performance and robustness against various watermark removal attacks and
ambiguity attack.
