  Federated Learning aims to train distributed deep models without sharing the
raw data with the centralized server. Similarly, in distributed inference of
neural networks, by partitioning the network and distributing it across several
physical nodes, activations and gradients are exchanged between physical nodes,
rather than raw data. Nevertheless, when a neural network is partitioned and
distributed among physical nodes, failure of physical nodes causes the failure
of the neural units that are placed on those nodes, which results in a
significant performance drop. Current approaches focus on resiliency of
training in distributed neural networks. However, resiliency of inference in
distributed neural networks is less explored. We introduce ResiliNet, a scheme
for making inference in distributed neural networks resilient to physical node
failures. ResiliNet combines two concepts to provide resiliency: skip
hyperconnection, a concept for skipping nodes in distributed neural networks
similar to skip connection in resnets, and a novel technique called failout,
which is introduced in this paper. Failout simulates physical node failure
conditions during training using dropout, and is specifically designed to
improve the resiliency of distributed neural networks. The results of the
experiments and ablation studies using three datasets confirm the ability of
ResiliNet to provide inference resiliency for distributed neural networks.
