  Annotating seismic data is expensive, laborious and subjective due to the
number of years required for seismic interpreters to attain proficiency in
interpretation. In this paper, we develop a framework to automate annotating
pixels of a seismic image to delineate geological structural elements given
image-level labels assigned to each image. Our framework factorizes the latent
space of a deep encoder-decoder network by projecting the latent space to
learned sub-spaces. Using constraints in the pixel space, the seismic image is
further factorized to reveal confidence values on pixels associated with the
geological element of interest. Details of the annotated image are provided for
analysis and qualitative comparison is made with similar frameworks.
