  Local windows are routinely used in computer vision and almost without
exception the center of the window is aligned with the pixels being processed.
We show that this conventional wisdom is not universally applicable. When a
pixel is on an edge, placing the center of the window on the pixel is one of
the fundamental reasons that cause many filtering algorithms to blur the edges.
Based on this insight, we propose a new Side Window Filtering (SWF) technique
which aligns the window's side or corner with the pixel being processed. The
SWF technique is surprisingly simple yet theoretically rooted and very
effective in practice. We show that many traditional linear and nonlinear
filters can be easily implemented under the SWF framework. Extensive analysis
and experiments show that implementing the SWF principle can significantly
improve their edge preserving capabilities and achieve state of the art
performances in applications such as image smoothing, denoising, enhancement,
structure-preserving texture-removing, mutual-structure extraction, and HDR
tone mapping. In addition to image filtering, we further show that the SWF
principle can be extended to other applications involving the use of a local
window. Using colorization by optimization as an example, we demonstrate that
implementing the SWF principle can effectively prevent artifacts such as color
leakage associated with the conventional implementation. Given the ubiquity of
window based operations in computer vision, the new SWF technique is likely to
benefit many more applications.
