  Recent advancements in deep convolutional neural networks have significantly
improved the performance of saliency prediction. However, the manual
configuration of the neural network architectures requires domain knowledge
expertise and can still be time-consuming and error-prone. To solve this, we
propose a new Neural Architecture Search (NAS) framework for saliency
prediction with two contributions. Firstly, a supernet for saliency prediction
is built with a weight-sharing network containing all candidate architectures,
by integrating a dynamic convolution into the encoder-decoder in the supernet,
termed SalNAS. Secondly, despite the fact that SalNAS is highly efficient
(20.98 million parameters), it can suffer from the lack of generalization. To
solve this, we propose a self-knowledge distillation approach, termed Self-KD,
that trains the student SalNAS with the weighted average information between
the ground truth and the prediction from the teacher model. The teacher model,
while sharing the same architecture, contains the best-performing weights
chosen by cross-validation. Self-KD can generalize well without the need to
compute the gradient in the teacher model, enabling an efficient training
system. By utilizing Self-KD, SalNAS outperforms other state-of-the-art
saliency prediction models in most evaluation rubrics across seven benchmark
datasets while being a lightweight model. The code will be available at
https://github.com/chakkritte/SalNAS
