  Studies of cosmology, galaxy evolution, and astronomical transients with
current and next-generation wide-field imaging surveys like the Rubin
Observatory Legacy Survey of Space and Time (LSST) are all critically dependent
on estimates of photometric redshifts. Capsule networks are a new type of
neural network architecture that is better suited for identifying morphological
features of the input images than traditional convolutional neural networks. We
use a deep capsule network trained on $ugriz$ images, spectroscopic redshifts,
and Galaxy Zoo spiral/elliptical classifications of $\sim$400,000 Sloan Digital
Sky Survey (SDSS) galaxies to do photometric redshift estimation. We achieve a
photometric redshift prediction accuracy and a fraction of catastrophic
outliers that are comparable to or better than current methods for SDSS main
galaxy sample-like data sets ($r\leq17.8$ and $z_{\mathrm{spec}}\leq0.4$) while
requiring less data and fewer trainable parameters. Furthermore, the
decision-making of our capsule network is much more easily interpretable as
capsules act as a low-dimensional encoding of the image. When the capsules are
projected on a 2-dimensional manifold, they form a single redshift sequence
with the fraction of spirals in a region exhibiting a gradient roughly
perpendicular to the redshift sequence. We perturb encodings of real galaxy
images in this low-dimensional space to create synthetic galaxy images that
demonstrate the image properties (e.g., size, orientation, and surface
brightness) encoded by each dimension. We also measure correlations between
galaxy properties (e.g., magnitudes, colours, and stellar mass) and each
capsule dimension. We publicly release our code, estimated redshifts, and
additional catalogues at https://biprateep.github.io/encapZulate-1 .
