DeepAI AI Chat
Log In Sign Up

Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization

03/09/2022
by   Sam Buchanan, et al.
5

Achieving invariance to nuisance transformations is a fundamental challenge in the construction of robust and reliable vision systems. Existing approaches to invariance scale exponentially with the dimension of the family of transformations, making them unable to cope with natural variabilities in visual data such as changes in pose and perspective. We identify a common limitation of these approaches–they rely on sampling to traverse the high-dimensional space of transformations–and propose a new computational primitive for building invariant networks based instead on optimization, which in many scenarios provides a provably more efficient method for high-dimensional exploration than sampling. We provide empirical and theoretical corroboration of the efficiency gains and soundness of our proposed method, and demonstrate its utility in constructing an efficient invariant network for a simple hierarchical object detection task when combined with unrolled optimization. Code for our networks and experiments is available at https://github.com/sdbuch/refine.

READ FULL TEXT

page 2

page 6

page 16

04/01/2014

A Deep Representation for Invariance And Music Classification

Representations in the auditory cortex might be based on mechanisms simi...
02/24/2017

How ConvNets model Non-linear Transformations

In this paper, we theoretically address three fundamental problems invol...
09/14/2016

Warped Convolutions: Efficient Invariance to Spatial Transformations

Convolutional Neural Networks (CNNs) are extremely efficient, since they...
10/16/2019

On the Global Optima of Kernelized Adversarial Representation Learning

Adversarial representation learning is a promising paradigm for obtainin...
03/13/2020

Random smooth gray value transformations for cross modality learning with gray value invariant networks

Random transformations are commonly used for augmentation of the trainin...
03/20/2023

EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning

Learning to predict agent motions with relationship reasoning is importa...
11/14/2019

Gradientless Descent: High-Dimensional Zeroth-Order Optimization

Zeroth-order optimization is the process of minimizing an objective f(x)...

Code Repositories

refine

Code for the paper "Resource Efficient Invariant Networks: Exponential Gains by Unrolled Optimization"


view repo