  The objective of dense material segmentation is to identify the material
categories for every image pixel. Recent studies adopt image patches to extract
material features. Although the trained networks can improve the segmentation
performance, their methods choose a fixed patch resolution which fails to take
into account the variation in pixel area covered by each material. In this
paper, we propose the Dynamic Backward Attention Transformer (DBAT) to
aggregate cross-resolution features. The DBAT takes cropped image patches as
input and gradually increases the patch resolution by merging adjacent patches
at each transformer stage, instead of fixing the patch resolution during
training. We explicitly gather the intermediate features extracted from
cross-resolution patches and merge them dynamically with predicted attention
masks. Experiments show that our DBAT achieves an accuracy of 86.85%, which is
the best performance among state-of-the-art real-time models. Like other
successful deep learning solutions with complex architectures, the DBAT also
suffers from lack of interpretability. To address this problem, this paper
examines the properties that the DBAT makes use of. By analysing the
cross-resolution features and the attention weights, this paper interprets how
the DBAT learns from image patches. We further align features to semantic
labels, performing network dissection, to infer that the proposed model can
extract material-related features better than other methods. We show that the
DBAT model is more robust to network initialisation, and yields fewer variable
predictions compared to other models. The project code is available at
https://github.com/heng-yuwen/Dynamic-Backward-Attention-Transformer.
