Exploiting the Distortion-Semantic Interaction in Fisheye Data

by   Kiran Kokilepersaud, et al.

In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality. Fisheye data has the wider field of view advantage over other types of cameras, but this comes at the expense of high radial distortion. As a result, objects further from the center exhibit deformations that make it difficult for a model to identify their semantic context. While previous work has attempted architectural and training augmentation changes to alleviate this effect, no work has attempted to guide the model towards learning a representation space that reflects this interaction between distortion and semantic context inherent to fisheye data. We introduce an approach to exploit this relationship by first extracting distortion class labels based on an object's distance from the center of the image. We then shape a backbone's representation space with a weighted contrastive loss that constrains objects of the same semantic class and distortion class to be close to each other within a lower dimensional embedding space. This backbone trained with both semantic and distortion information is then fine-tuned within an object detection setting to empirically evaluate the quality of the learnt representation. We show this method leads to performance improvements by as much as 1.1 detection strategies and .6 representation learning approaches.


page 3

page 8

page 11


Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection

Building object detectors that are robust to domain shifts is critical f...

Single View Distortion Correction using Semantic Guidance

Most distortion correction methods focus on simple forms of distortion, ...

Rational Radial Distortion Models with Analytical Undistortion Formulae

The common approach to radial distortion is by the means of polynomial a...

Learning the Space of Deep Models

Embedding of large but redundant data, such as images or text, in a hier...

Unifying Voxel-based Representation with Transformer for 3D Object Detection

In this work, we present a unified framework for multi-modality 3D objec...

Clinically Labeled Contrastive Learning for OCT Biomarker Classification

This paper presents a novel positive and negative set selection strategy...

Please sign up or login with your details

Forgot password? Click here to reset