  Very few methods for hybrid federated learning, where clients only hold
subsets of both features and samples, exist. Yet, this scenario is extremely
important in practical settings. We provide a fast, robust algorithm for hybrid
federated learning that hinges on Fenchel Duality. We prove the convergence of
the algorithm to the same solution as if the model is trained centrally in a
variety of practical regimes. Furthermore, we provide experimental results that
demonstrate the performance improvements of the algorithm over a commonly used
method in federated learning, FedAvg, and an existing hybrid FL algorithm,
HyFEM. We also provide privacy considerations and necessary steps to protect
client data.
