DarSwin: Distortion Aware Radial Swin Transformer

by   Akshaya Athwale, et al.

Wide-angle lenses are commonly used in perception tasks requiring a large field of view. Unfortunately, these lenses produce significant distortions making conventional models that ignore the distortion effects unable to adapt to wide-angle images. In this paper, we present a novel transformer-based model that automatically adapts to the distortion produced by wide-angle lenses. We leverage the physical characteristics of such lenses, which are analytically defined by the radial distortion profile (assumed to be known), to develop a distortion aware radial swin transformer (DarSwin). In contrast to conventional transformer-based architectures, DarSwin comprises a radial patch partitioning, a distortion-based sampling technique for creating token embeddings, and a polar position encoding for radial patch merging. We validate our method on classification tasks using synthetically distorted ImageNet data and show through extensive experiments that DarSwin can perform zero-shot adaptation to unseen distortions of different wide-angle lenses. Compared to other baselines, DarSwin achieves the best results (in terms of Top-1 and -5 accuracy), when tested on in-distribution data, with almost 2 under medium (high) distortion levels, and comparable to the state-of-the-art under low and very low distortion levels (perspective-like images).


page 1

page 4

page 6


Rational Radial Distortion Models with Analytical Undistortion Formulae

The common approach to radial distortion is by the means of polynomial a...

An Analytical Piecewise Radial Distortion Model for Precision Camera Calibration

The common approach to radial distortion is by the means of polynomial a...

FishFormer: Annulus Slicing-based Transformer for Fisheye Rectification with Efficacy Domain Exploration

Numerous significant progress on fisheye image rectification has been ac...

3D Reconstruction with Low Resolution, Small Baseline and High Radial Distortion Stereo Images

In this paper we analyze and compare approaches for 3D reconstruction fr...

A Family of Simplified Geometric Distortion Models for Camera Calibration

The commonly used radial distortion model for camera calibration is in f...

RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning

The wide-angle lens shows appealing applications in VR technologies, but...

Non-uniform Sampling Strategies for NeRF on 360° images

In recent years, the performance of novel view synthesis using perspecti...

Please sign up or login with your details

Forgot password? Click here to reset