Valerii Likhosherstov | DeepAI

DeepAI AI Chat

Featured Co-authors

Honglak Lee
129 publications
Yuan Gao
112 publications
Adrian Weller
101 publications
Tianyi Zhang
86 publications
Yiming Yang
76 publications
Yi Tay
76 publications
Di He
71 publications
Krzysztof Choromanski
63 publications
Michael S. Ryoo
53 publications
Aldo Pacchiano
53 publications
Vikas Sindhwani
45 publications

∙ 02/03/2023

Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers

We propose a new class of linear Transformers called FourierLearner-Tran...

0 Krzysztof Marcin Choromanski, et al. ∙

∙ 02/02/2023

Efficient Graph Field Integrators Meet Point Clouds

We present two new classes of algorithms for efficient field integration...

0 Krzysztof Choromanski, et al. ∙

∙ 02/01/2023

FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Features

The problem of efficient approximation of a linear operator induced by t...

0 Valerii Likhosherstov, et al. ∙

∙ 01/31/2023

Simplex Random Features

We present Simplex Random Features (SimRFs), a new random feature (RF) m...

0 Isaac Reid, et al. ∙

∙ 05/30/2022

Chefs' Random Tables: Non-Trigonometric Random Features

We introduce chefs' random tables (CRTs), a new class of non-trigonometr...

0 Valerii Likhosherstov, et al. ∙

∙ 11/25/2021

PolyViT: Co-training Vision Transformers on Images, Videos and Audio

Can we train a single transformer model capable of processing multiple m...

5 Valerii Likhosherstov, et al. ∙

∙ 10/08/2021

Hybrid Random Features

We propose a new class of random feature methods for linearizing softmax...

0 Krzysztof Choromanski, et al. ∙

∙ 06/07/2021

On the Expressive Power of Self-Attention Matrices

Transformer networks are able to capture patterns in data coming from ma...

0 Valerii Likhosherstov, et al. ∙

∙ 06/04/2021

Debiasing a First-order Heuristic for Approximate Bi-level Optimization

Approximate bi-level optimization (ABLO) consists of (outer-level) optim...

0 Valerii Likhosherstov, et al. ∙

∙ 02/08/2021

Unlocking Pixels for Reinforcement Learning via Implicit Attention

There has recently been significant interest in training reinforcement l...

1 Krzysztof Choromanski, et al. ∙

∙ 12/21/2020

Sub-Linear Memory: How to Make Performers SLiM

The Transformer architecture has revolutionized deep learning on sequent...

0 Valerii Likhosherstov, et al. ∙

∙ 09/30/2020

Rethinking Attention with Performers

We introduce Performers, Transformer architectures which can estimate re...

5 Krzysztof Choromanski, et al. ∙

∙ 06/19/2020

An Ode to an ODE

We present a new paradigm for Neural ODE algorithms, calledODEtoODE, whe...

18 Krzysztof Choromanski, et al. ∙

∙ 06/05/2020

UFO-BLO: Unbiased First-Order Bilevel Optimization

Bilevel optimization (BLO) is a popular approach with many applications ...

7 Valerii Likhosherstov, et al. ∙

∙ 06/05/2020

Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers

Transformer models have achieved state-of-the-art results across a diver...

10 Krzysztof Choromanski, et al. ∙

∙ 04/18/2020

CWY Parametrization for Scalable Learning of Orthogonal and Stiefel Matrices

In this paper we propose a new approach for optimization over orthogonal...

2 Valerii Likhosherstov, et al. ∙

∙ 03/30/2020

Stochastic Flows and Geometric Optimization on the Orthogonal Group

We present a new class of stochastic, geometrically-driven optimization ...

2 Krzysztof Choromanski, et al. ∙

∙ 10/22/2019

Tractable Minor-free Generalization of Planar Zero-field Ising Models

We present a new family of zero-field Ising models over N binary variabl...

0 Valerii Likhosherstov, et al. ∙

∙ 06/14/2019

A New Family of Tractable Ising Models

We present a new family of zero-field Ising models over N binary variabl...

0 Valerii Likhosherstov, et al. ∙

∙ 12/22/2018

Inference and Sampling of K_33-free Ising Models

We call an Ising model tractable when it is possible to compute its part...

0 Valerii Likhosherstov, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world