  State-of-the-art video deblurring methods cannot handle blurry videos
recorded in dynamic scenes, since they are built under a strong assumption that
the captured scenes are static. Contrary to the existing methods, we propose a
video deblurring algorithm that can deal with general blurs inherent in dynamic
scenes. To handle general and locally varying blurs caused by various sources,
such as moving objects, camera shake, depth variation, and defocus, we estimate
pixel-wise non-uniform blur kernels. We infer bidirectional optical flows to
handle motion blurs, and also estimate Gaussian blur maps to remove optical
blur from defocus in our new blur model. Therefore, we propose a single energy
model that jointly estimates optical flows, defocus blur maps and latent
frames. We also provide a framework and efficient solvers to minimize the
proposed energy model. By optimizing the energy model, we achieve significant
improvements in removing general blurs, estimating optical flows, and extending
depth-of-field in blurry frames. Moreover, in this work, to evaluate the
performance of non-uniform deblurring methods objectively, we have constructed
a new realistic dataset with ground truths. In addition, extensive experimental
on publicly available challenging video data demonstrate that the proposed
method produces qualitatively superior performance than the state-of-the-art
methods which often fail in either deblurring or optical flow estimation.
