Spherical Transformer

02/10/2022
by   Sungmin Cho, et al.
0

Using convolutional neural networks for 360images can induce sub-optimal performance due to distortions entailed by a planar projection. The distortion gets deteriorated when a rotation is applied to the 360image. Thus, many researches based on convolutions attempt to reduce the distortions to learn accurate representation. In contrast, we leverage the transformer architecture to solve image classification problems for 360images. Using the proposed transformer for 360images has two advantages. First, our method does not require the erroneous planar projection process by sampling pixels from the sphere surface. Second, our sampling method based on regular polyhedrons makes low rotation equivariance errors, because specific rotations can be reduced to permutations of faces. In experiments, we validate our network on two aspects, as follows. First, we show that using a transformer with highly uniform sampling methods can help reduce the distortion. Second, we demonstrate that the transformer architecture can achieve rotation equivariance on specific rotations. We compare our method to other state-of-the-art algorithms using the SPH-MNIST, SPH-CIFAR, and SUN360 datasets and show that our method is competitive with other methods.

READ FULL TEXT

page 1

page 3

page 6

research
01/11/2021

Spherical Transformer: Adapting Spherical Signal to ConvolutionalNetworks

Convolutional neural networks (CNNs) have been widely used in various vi...
research
08/28/2023

PanoSwin: a Pano-style Swin Transformer for Panorama Understanding

In panorama understanding, the widely used equirectangular projection (E...
research
07/14/2023

HEAL-SWIN: A Vision Transformer On The Sphere

High-resolution wide-angle fisheye images are becoming more and more imp...
research
09/06/2017

Polar Transformer Networks

Convolutional neural networks (CNNs) are inherently equivariant to trans...
research
08/02/2017

Flat2Sphere: Learning Spherical Convolution for Fast Features from 360° Imagery

While 360 cameras offer tremendous new possibilities in vision, graphics...
research
07/10/2023

Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer

Constraint satisfaction problems (CSPs) are about finding values of vari...

Please sign up or login with your details

Forgot password? Click here to reset