  The use of deep learning techniques for automatic facial expression
recognition has recently attracted great interest but developed models are
still unable to generalize well due to the lack of large emotion datasets for
deep learning. To overcome this problem, in this paper, we propose utilizing a
novel transfer learning approach relying on PathNet and investigate how
knowledge can be accumulated within a given dataset and how the knowledge
captured from one emotion dataset can be transferred into another in order to
improve the overall performance. To evaluate the robustness of our system, we
have conducted various sets of experiments on two emotion datasets: SAVEE and
eNTERFACE. The experimental results demonstrate that our proposed system leads
to improvement in performance of emotion recognition and performs significantly
better than the recent state-of-the-art schemes adopting fine-\
tuning/pre-trained approaches.
