  Spectral embedding provides a framework for solving perceptual organization
problems, including image segmentation and figure/ground organization. From an
affinity matrix describing pairwise relationships between pixels, it clusters
pixels into regions, and, using a complex-valued extension, orders pixels
according to layer. We train a convolutional neural network (CNN) to directly
predict the pairwise relationships that define this affinity matrix. Spectral
embedding then resolves these predictions into a globally-consistent
segmentation and figure/ground organization of the scene. Experiments
demonstrate significant benefit to this direct coupling compared to prior works
which use explicit intermediate stages, such as edge detection, on the pathway
from image to affinities. Our results suggest spectral embedding as a powerful
alternative to the conditional random field (CRF)-based globalization schemes
typically coupled to deep neural networks.
