  Training deep neural networks is known to require a large number of training
samples. However, in many applications only few training samples are available.
In this work, we tackle the issue of training neural networks for
classification task when few training samples are available. We attempt to
solve this issue by proposing a new regularization term that constrains the
hidden layers of a network to learn class-wise invariant representations. In
our regularization framework, learning invariant representations is generalized
to the class membership where samples with the same class should have the same
representation. Numerical experiments over MNIST and its variants showed that
our proposal helps improving the generalization of neural network particularly
when trained with few samples. We provide the source code of our framework
https://github.com/sbelharbi/learning-class-invariant-features .
