  We propose a new algorithm to learn a one-hidden-layer convolutional neural
network where both the convolutional weights and the outputs weights are
parameters to be learned. Our algorithm works for a general class of
(potentially overlapping) patches, including commonly used structures for
computer vision tasks. Our algorithm draws ideas from (1) isotonic regression
for learning neural networks and (2) landscape analysis of non-convex matrix
factorization problems. We believe these findings may inspire further
development in designing provable algorithms for learning neural networks and
other complex models.
