  In this paper, we aim at automatically searching an efficient network
architecture for dense image prediction. Particularly, we follow the
encoder-decoder style and focus on designing a connectivity structure for the
decoder. To achieve that, we design a densely connected network with learnable
connections, named Fully Dense Network, which contains a large set of possible
final connectivity structures. We then employ gradient descent to search the
optimal connectivity from the dense connections. The search process is guided
by a novel loss function, which pushes the weight of each connection to be
binary and the connections to be sparse. The discovered connectivity achieves
competitive results on two segmentation datasets, while runs more than three
times faster and requires less than half parameters compared to the
state-of-the-art methods. An extensive experiment shows that the discovered
connectivity is compatible with various backbones and generalizes well to other
dense image prediction tasks.
