  Federated learning is a promising machine learning technique that enables
multiple clients to collaboratively build a model without revealing the raw
data to each other. Among various types of federated learning methods,
horizontal federated learning (HFL) is the best-studied category and handles
homogeneous feature spaces. However, in the case of heterogeneous feature
spaces, HFL uses only common features and leaves client-specific features
unutilized. In this paper, we propose a HFL method using neural networks named
continual horizontal federated learning (CHFL), a continual learning approach
to improve the performance of HFL by taking advantage of unique features of
each client. CHFL splits the network into two columns corresponding to common
features and unique features, respectively. It jointly trains the first column
by using common features through vanilla HFL and locally trains the second
column by using unique features and leveraging the knowledge of the first one
via lateral connections without interfering with the federated training of it.
We conduct experiments on various real world datasets and show that CHFL
greatly outperforms vanilla HFL that only uses common features and local
learning that uses all features that each client has.
