  Wide-angle lenses are commonly used in perception tasks requiring a large
field of view. Unfortunately, these lenses produce significant distortions,
making conventional models that ignore the distortion effects unable to adapt
to wide-angle images. In this paper, we present a novel transformer-based model
that automatically adapts to the distortion produced by wide-angle lenses. Our
proposed image encoder architecture, dubbed DarSwin, leverages the physical
characteristics of such lenses analytically defined by the radial distortion
profile. In contrast to conventional transformer-based architectures, DarSwin
comprises a radial patch partitioning, a distortion-based sampling technique
for creating token embeddings, and an angular position encoding for radial
patch merging. Compared to other baselines, DarSwin achieves the best results
on different datasets with significant gains when trained on bounded levels of
distortions (very low, low, medium, and high) and tested on all, including
out-of-distribution distortions. While the base DarSwin architecture requires
knowledge of the radial distortion profile, we show it can be combined with a
self-calibration network that estimates such a profile from the input image
itself, resulting in a completely uncalibrated pipeline. Finally, we also
present DarSwin-Unet, which extends DarSwin, to an encoder-decoder architecture
suitable for pixel-level tasks. We demonstrate its performance on depth
estimation and show through extensive experiments that DarSwin-Unet can perform
zero-shot adaptation to unseen distortions of different wide-angle lenses. The
code and models are publicly available at https://lvsn.github.io/darswin/
