  Differentiable programming is able to combine different functions or programs
in a processing pipeline with the goal of applying end-to-end learning or
optimization. A significant impediment is the non-differentiable nature of some
algorithms. We propose to use synthetic gradients (SG) to overcome this
difficulty. SG uses the universal function approximation property of neural
networks. We apply SG to combine convolutional neural network (CNN) with
dynamic programming (DP) in end-to-end learning for segmenting left ventricle
from short axis view of heart MRI. Our experiments show that end-to-end
combination of CNN and DP requires fewer labeled images to achieve a
significantly better segmentation accuracy than using only CNN.
