Andrej Risteski

research

∙ 06/15/2023

Fit Like You Sample: Sample-Efficient Generalized Score Matching from Fast Mixing Markov Chains

Score matching is an approach to learning probability distributions para...

0 Yilong Qin, et al. ∙

research

∙ 06/03/2023

Provable benefits of score matching

Score matching is an alternative to maximum likelihood (ML) for estimati...

0 Chirag Pabbaraju, et al. ∙

research

∙ 06/01/2023

Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation

Good data augmentation is one of the key factors that lead to the empiri...

0 Runtian Zhai, et al. ∙

research

∙ 03/07/2023

How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding

While the successes of transformers across many domains are indisputable...

0 Yuchen Li, et al. ∙

research

∙ 10/21/2022

Neural Network Approximations of PDEs Beyond Linearity: Representational Perspective

A burgeoning line of research has developed deep neural networks capable...

0 Tanya Marwah, et al. ∙

research

∙ 10/03/2022

Statistical Efficiency of Score Matching: The View from Isoperimetry

Deep generative models parametrized up to a normalizing constant (e.g. e...

0 Frederic Koehler, et al. ∙

research

∙ 10/01/2022

Pitfalls of Gaussians as a noise distribution in NCE

Noise Contrastive Estimation (NCE) is a popular approach for learning pr...

0 Holden Lee, et al. ∙

research

∙ 03/29/2022

Contrasting the landscape of contrastive and non-contrastive learning

A lot of recent advances in unsupervised feature learning are based on d...

4 Ashwini Pokle, et al. ∙

research

∙ 03/27/2022

Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions

Continual learning is an emerging paradigm in machine learning, wherein ...

0 Binghui Peng, et al. ∙

research

∙ 02/18/2022

Masked prediction tasks: a parameter identifiability view

The vast majority of work in self-supervised learning, both theoretical ...

0 Bingbin Liu, et al. ∙

research

∙ 02/17/2022

Sampling Approximately Low-Rank Ising Models: MCMC meets Variational Methods

We consider Ising models on the hypercube with a general interaction mat...

0 Frederic Koehler, et al. ∙

research

∙ 02/14/2022

Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalization

A common explanation for the failure of deep networks to generalize out-...

0 Elan Rosenfeld, et al. ∙

research

∙ 12/13/2021

Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias

Variational Autoencoders (VAEs) are one of the most commonly used genera...

0 Frederic Koehler, et al. ∙

research

∙ 10/21/2021

Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation

Noise-contrastive estimation (NCE) is a statistically consistent method ...

0 Bingbin Liu, et al. ∙

research

∙ 07/09/2021

The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders

Training and using modern neural-network based latent-variable generativ...

0 Divyansh Pareek, et al. ∙

research

∙ 07/07/2021

Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flows

Normalizing flows are a widely used class of latent-variable generative ...

0 Holden Lee, et al. ∙

research

∙ 06/18/2021

Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

Domain generalization aims at performing well on unseen test environment...

0 Yining Chen, et al. ∙

research

∙ 06/03/2021

The Limitations of Limited Context for Constituency Parsing

Incorporating syntax into neural approaches in NLP has a multitude of pr...

6 Yuchen Li, et al. ∙

research

∙ 03/03/2021

Contrastive learning of strong-mixing continuous-time stochastic processes

Contrastive learning is a family of self-supervised methods where a mode...

0 Bingbin Liu, et al. ∙

research

∙ 03/03/2021

Parametric Complexity Bounds for Approximating PDEs with Neural Networks

Recent empirical results show that deep networks can approximate solutio...

0 Tanya Marwah, et al. ∙

research

∙ 02/25/2021

An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization

A popular assumption for out-of-distribution generalization is that the ...

0 Elan Rosenfeld, et al. ∙

research

∙ 10/12/2020

The Risks of Invariant Risk Minimization

Invariant Causal Prediction (Peters et al., 2016) is a technique for out...

0 Elan Rosenfeld, et al. ∙

research

∙ 10/02/2020

Representational aspects of depth and conditioning in normalizing flows

Normalizing flows are among the most popular paradigms in generative mod...

11 Frederic Koehler, et al. ∙

research

∙ 09/30/2020

Efficient sampling from the Bingham distribution

We give a algorithm for exact sampling from the Bingham distribution p(x...

0 Rong Ge, et al. ∙

research

∙ 08/11/2020

On Learning Language-Invariant Representations for Universal Machine Translation

The goal of universal machine translation is to learn to translate betwe...

38 Han Zhao, et al. ∙

research

∙ 02/13/2020

Fast Convergence for Langevin Diffusion with Matrix Manifold Structure

In this paper, we study the problem of sampling from distributions of th...

0 Ankur Moitra, et al. ∙

research

∙ 06/28/2019

Benefits of Overparameterization in Single-Layer Latent Variable Generative Models

One of the most surprising and exciting discoveries in supervising learn...

1 Rares-Darius Buhai, et al. ∙

research

∙ 05/30/2019

Sum-of-squares meets square loss: Fast rates for agnostic tensor completion

We study tensor completion in the agnostic setting. In the classical ten...

0 Dylan J. Foster, et al. ∙

research

∙ 11/29/2018

Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition

A key task in Bayesian machine learning is sampling from distributions t...

0 Rong Ge, et al. ∙

research

∙ 08/22/2018

Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective

The free energy is a key quantity of interest in Ising models, but unfor...

0 Vishesh Jain, et al. ∙

research

∙ 06/27/2018

Approximability of Discriminators Implies Diversity in GANs

While Generative Adversarial Networks (GANs) have empirically produced i...

0 Yu Bai, et al. ∙

research

∙ 05/29/2018

Representational Power of ReLU Networks and Polynomial Kernels: Beyond Worst-Case Analysis

There has been a large amount of interest, both in the past and particul...

0 Frederic Koehler, et al. ∙

research

∙ 11/07/2017

Theoretical limitations of Encoder-Decoder GAN architectures

Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an ...

0 Sanjeev Arora, et al. ∙

research

∙ 10/07/2017

Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo

A key task in Bayesian statistics is sampling from distributions that ar...

0 Rong Ge, et al. ∙

research

∙ 06/14/2017

Provable benefits of representation learning

There is general consensus that learning representations is useful for a...

0 Sanjeev Arora, et al. ∙

research

∙ 04/29/2017

Extending and Improving Wordnet via Unsupervised Word Embeddings

This work presents an unsupervised approach for improving WordNet that b...

0 Mikhail Khodak, et al. ∙

research

∙ 12/28/2016

Provable learning of Noisy-or Networks

Many machine learning applications use latent variable models to explain...

0 Sanjeev Arora, et al. ∙

research

∙ 11/11/2016

Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates

Non-negative matrix factorization is a popular tool for decomposing data...

0 Yuanzhi Li, et al. ∙

research

∙ 07/12/2016

Approximate maximum entropy principles via Goemans-Williamson with applications to provable variational methods

The well known maximum-entropy principle due to Jaynes, which states tha...

0 Yuanzhi Li, et al. ∙

research

∙ 07/11/2016

How to calculate partition functions using convex programming hierarchies: provable bounds for variational methods

We consider the problem of approximating partition functions for Ising m...

0 Andrej Risteski, et al. ∙

research

∙ 02/06/2016

Recovery guarantee of weighted low-rank approximation via alternating minimization

Many applications require recovering a ground truth low-rank matrix from...

0 Yuanzhi Li, et al. ∙

research

∙ 01/14/2016

Linear Algebraic Structure of Word Senses, with Applications to Polysemy

Word embeddings are ubiquitous in NLP and information retrieval, but it'...

0 Sanjeev Arora, et al. ∙

research

∙ 03/23/2015

On some provably correct cases of variational inference for topic models

Variational inference is a very efficient and popular heuristic used in ...

0 Pranjal Awasthi, et al. ∙

research

∙ 02/12/2015

RAND-WALK: A Latent Variable Model Approach to Word Embeddings

Semantic word embeddings represent the meaning of a word via a vector, a...

0 Sanjeev Arora, et al. ∙

Andrej Risteski

Featured Co-authors