  We present a conceptually simple but effective funnel activation for image
recognition tasks, called Funnel activation (FReLU), that extends ReLU and
PReLU to a 2D activation by adding a negligible overhead of spatial condition.
The forms of ReLU and PReLU are y = max(x, 0) and y = max(x, px), respectively,
while FReLU is in the form of y = max(x,T(x)), where T(x) is the 2D spatial
condition. Moreover, the spatial condition achieves a pixel-wise modeling
capacity in a simple way, capturing complicated visual layouts with regular
convolutions. We conduct experiments on ImageNet, COCO detection, and semantic
segmentation tasks, showing great improvements and robustness of FReLU in the
visual recognition tasks. Code is available at
https://github.com/megvii-model/FunnelAct.
