  We present HDP-VFL, the first hybrid differentially private (DP) framework
for vertical federated learning (VFL) to demonstrate that it is possible to
jointly learn a generalized linear model (GLM) from vertically partitioned data
with only a negligible cost, w.r.t. training time, accuracy, etc., comparing to
idealized non-private VFL. Our work builds on the recent advances in VFL-based
collaborative training among different organizations which rely on protocols
like Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC) to
secure computation and training. In particular, we analyze how VFL's
intermediate result (IR) can leak private information of the training data
during communication and design a DP-based privacy-preserving algorithm to
ensure the data confidentiality of VFL participants. We mathematically prove
that our algorithm not only provides utility guarantees for VFL, but also
offers multi-level privacy, i.e. DP w.r.t. IR and joint differential privacy
(JDP) w.r.t. model weights. Experimental results demonstrate that our work,
under adequate privacy budgets, is quantitatively and qualitatively similar to
GLMs, learned in idealized non-private VFL setting, rather than the increased
cost in memory and processing time in most prior works based on HE or MPC. Our
codes will be released if this paper is accepted.
