  Computer vision can be understood as the ability to perform inference on
image data. Breakthroughs in computer vision technology are often marked by
advances in inference techniques. This thesis proposes novel inference schemes
and demonstrates applications in computer vision. We propose inference
techniques for both generative and discriminative vision models. The use of
generative models in vision is often hampered by the difficulty of posterior
inference. We propose techniques for improving inference in MCMC sampling and
message-passing inference. Our inference strategy is to learn separate
discriminative models that assist Bayesian inference in a generative model.
Experiments on a range of generative models show that the proposed techniques
accelerate the inference process and/or converge to better solutions. A main
complication in the design of discriminative models is the inclusion of prior
knowledge. We concentrate on CNN models and propose a generalization of
standard spatial convolutions to bilateral convolutions. We generalize the
existing use of bilateral filters and then propose new neural network
architectures with learnable bilateral filters, which we call `Bilateral Neural
Networks'. Experiments demonstrate the use of the bilateral networks on a wide
range of image and video tasks and datasets. In summary, we propose techniques
for better inference in several vision models ranging from inverse graphics to
freely parameterized neural networks. In generative models, our inference
techniques alleviate some of the crucial hurdles in Bayesian posterior
inference, paving new ways for the use of model based machine learning in
vision. In discriminative CNN models, the proposed filter generalizations aid
in the design of new neural network architectures that can handle sparse
high-dimensional data as well as provide a way to incorporate prior knowledge
into CNNs.
