  Spiking neural networks (SNNs) are rich in spatio-temporal dynamics and are
suitable for processing event-based neuromorphic data. However, event-based
datasets are usually less annotated than static datasets. This small data scale
makes SNNs prone to overfitting and limits their performance. In order to
improve the generalization ability of SNNs on event-based datasets, we use
static images to assist SNN training on event data. In this paper, we first
discuss the domain mismatch problem encountered when directly transferring
networks trained on static datasets to event data. We argue that the
inconsistency of feature distributions becomes a major factor hindering the
effective transfer of knowledge from static images to event data. To address
this problem, we propose solutions in terms of two aspects: feature
distribution and training strategy. Firstly, we propose a knowledge transfer
loss, which consists of domain alignment loss and spatio-temporal
regularization. The domain alignment loss learns domain-invariant spatial
features by reducing the marginal distribution distance between the static
image and the event data. Spatio-temporal regularization provides dynamically
learnable coefficients for domain alignment loss by using the output features
of the event data at each time step as a regularization term. In addition, we
propose a sliding training strategy, which gradually replaces static image
inputs probabilistically with event data, resulting in a smoother and more
stable training for the network. We validate our method on neuromorphic
datasets, including N-Caltech101, CEP-DVS, and N-Omniglot. The experimental
results show that our proposed method achieves better performance on all
datasets compared to the current state-of-the-art methods. Code is available at
https://github.com/Brain-Cog-Lab/Transfer-for-DVS.
