  Selection of an architectural prior well suited to a task (e.g. convolutions
for image data) is crucial to the success of deep neural networks (NNs).
Conversely, the weight priors within these architectures are typically left
vague, e.g.~independent Gaussian distributions, which has led to debate over
the utility of Bayesian deep learning. This paper explores the benefits of
adding structure to weight priors. It initially considers first-layer filters
of a convolutional NN, designing a prior based on random Gabor filters. Second,
it considers adding structure to the prior of final-layer weights by estimating
how each hidden feature relates to each class. Empirical results suggest that
these structured weight priors lead to more meaningful functional priors for
image data. This contributes to the ongoing discussion on the importance of
weight priors.
