  Multi-level features are important for saliency detection. Better combination
and use of multi-level features with time information can greatly improve the
accuracy of the video saliency model. In order to fully combine multi-level
features and make it serve the video saliency model, we propose a 3D fully
convolutional encoder-decoder architecture for video saliency detection, which
combines scale, space and time information for video saliency modeling. The
encoder extracts multi-scale temporal-spatial features from the input
continuous video frames, and then constructs temporal-spatial feature pyramid
through temporal-spatial convolution and top-down feature integration. The
decoder performs hierarchical decoding of temporal-spatial features from
different scales, and finally produces a saliency map from the integration of
multiple video frames. Our model is simple yet effective, and can run in real
time. We perform abundant experiments, and the results indicate that the
well-designed structure can improve the precision of video saliency detection
significantly. Experimental results on three purely visual video saliency
benchmarks and six audio-video saliency benchmarks demonstrate that our method
outperforms the existing state-of-the-art methods.
