  This paper introduces EmMark,a novel watermarking framework for protecting
the intellectual property (IP) of embedded large language models deployed on
resource-constrained edge devices. To address the IP theft risks posed by
malicious end-users, EmMark enables proprietors to authenticate ownership by
querying the watermarked model weights and matching the inserted signatures.
EmMark's novelty lies in its strategic watermark weight parameters selection,
nsuring robustness and maintaining model quality. Extensive proof-of-concept
evaluations of models from OPT and LLaMA-2 families demonstrate EmMark's
fidelity, achieving 100% success in watermark extraction with model performance
preservation. EmMark also showcased its resilience against watermark removal
and forging attacks.
