  In this paper, to remedy this deficiency, we propose a Linear Attention
Mechanism which is approximate to dot-product attention with much less memory
and computational costs. The efficient design makes the incorporation between
attention mechanisms and neural networks more flexible and versatile.
Experiments conducted on semantic segmentation demonstrated the effectiveness
of linear attention mechanism. Code is available at
https://github.com/lironui/Linear-Attention-Mechanism.
