  Recent neural-network-based architectures for image segmentation make
extensive usage of feature forwarding mechanisms to integrate information from
multiple scales. Although yielding good results, even deeper architectures and
alternative methods for feature fusion at different resolutions have been
scarcely investigated for medical applications. In this work we propose to
implement segmentation via an encoder-decoder architecture which differs from
any other previously published method since (i) it employs a very deep
architecture based on residual learning and (ii) combines features via a
convolutional Long Short Term Memory (LSTM), instead of concatenation or
summation. The intuition is that the memory mechanism implemented by LSTMs can
better integrate features from different scales through a coarse-to-fine
strategy; hence the name Coarse-to-Fine Context Memory (CFCM). We demonstrate
the remarkable advantages of this approach on two datasets: the Montgomery
county lung segmentation dataset, and the EndoVis 2015 challenge dataset for
surgical instrument segmentation.
