  Convolutional neural networks are the way to solve arbitrary image
segmentation tasks. However, when images are large, memory demands often exceed
the available resources, in particular on a common GPU. Especially in
biomedical imaging, where 3D images are common, the problems are apparent. A
typical approach to solve this limitation is to break the task into smaller
subtasks by dividing images into smaller image patches. Another approach, if
applicable, is to look at the 2D image sections separately, and to solve the
problem in 2D. Often, the loss of global context makes such approaches less
effective; important global information might not be present in the current
image patch, or the selected 2D image section. Here, we propose Deep Neural
Patchworks (DNP), a segmentation framework that is based on hierarchical and
nested stacking of patch-based networks that solves the dilemma between global
context and memory limitations.
