  This work is the first attempt to evaluate and compare felderated learning
(FL) and split neural networks (SplitNN) in real-world IoT settings in terms of
learning performance and device implementation overhead. We consider a variety
of datasets, different model architectures, multiple clients, and various
performance metrics. For learning performance, which is specified by the model
accuracy and convergence speed metrics, we empirically evaluate both FL and
SplitNN under different types of data distributions such as imbalanced and
non-independent and identically distributed (non-IID) data. We show that the
learning performance of SplitNN is better than FL under an imbalanced data
distribution, but worse than FL under an extreme non-IID data distribution. For
implementation overhead, we end-to-end mount both FL and SplitNN on Raspberry
Pis, and comprehensively evaluate overheads including training time,
communication overhead under the real LAN setting, power consumption and memory
usage. Our key observations are that under IoT scenario where the communication
traffic is the main concern, the FL appears to perform better over SplitNN
because FL has the significantly lower communication overhead compared with
SplitNN, which empirically corroborate previous statistical analysis. In
addition, we reveal several unrecognized limitations about SplitNN, forming the
basis for future research.
