  We present results exploring the role that probabilistic deep learning models
can play in cosmology from large-scale astronomical surveys through photometric
redshift (photo-z) estimation. Photo-z uncertainty estimates are critical for
the science goals of upcoming large-scale surveys such as LSST, however common
machine learning methods typically provide only point estimates and lack
uncertainties on predictions. We turn to Bayesian neural networks (BNNs) as a
promising way to provide accurate predictions of redshift values with
uncertainty estimates. We have compiled a galaxy data set from the Hyper
Suprime-Cam Survey with grizy photometry, which is designed to be a smaller
scale version of large surveys like LSST. We use this data set to investigate
the performance of a neural network (NN) and a probabilistic BNN for photo-z
estimation and evaluate their performance with respect to LSST photo-z science
requirements. We also examine the utility of photo-z uncertainties as a means
to reduce catastrophic outlier estimates. The BNN outputs the estimate in the
form of a Gaussian probability distribution. We use the mean and standard
deviation as the redshift estimate and uncertainty. We find that the BNN can
produce accurate uncertainties. Using a coverage test, we find excellent
agreement with expectation -- 67.2$\%$ of galaxies between $0 < 2.5$ have
1-$\sigma$ uncertainties that cover the spectroscopic value. We also include a
comparison to alternative machine learning models using the same data. We find
the BNN meets two out of three of the LSST photo-z science requirements in the
range $0 < z < 2.5$.
