b'Tengyu Ma'

research

∙ 09/07/2023

Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation

Object detection in low-light scenarios has attracted much attention in ...

0 Xiaohan Cui, et al. ∙

research

∙ 07/20/2023

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization

Despite extensive studies, the underlying reason as to why overparameter...

0 Kaiyue Wen, et al. ∙

research

∙ 07/07/2023

One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention

Recent works have empirically analyzed in-context learning and shown tha...

0 Arvind Mahankali, et al. ∙

research

∙ 06/28/2023

Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time

Despite recent theoretical progress on the non-convex optimization of tw...

0 Arvind Mahankali, et al. ∙

research

∙ 06/22/2023

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization

Recent works on over-parameterized neural networks have shown that the s...

0 Khashayar Gatmiry, et al. ∙

research

∙ 05/26/2023

Large Language Models as Tool Makers

Recent research shows the potential of enhancing the problem-solving abi...

7 Tianle Cai, et al. ∙

research

∙ 05/23/2023

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Given the massive cost of language model pre-training, a non-trivial imp...

0 Hong Liu, et al. ∙

research

∙ 05/17/2023

DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining

The mixture proportions of pretraining data domains (e.g., Wikipedia, bo...

0 Sang Michael Xie, et al. ∙

research

∙ 05/15/2023

Symbol tuning improves in-context learning in language models

We present symbol tuning - finetuning language models on in-context inpu...

0 Jerry Wei, et al. ∙

research

∙ 04/29/2023

Toward L_∞-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields

Many machine learning applications require learning a function with a sm...

0 Kefan Dong, et al. ∙

research

∙ 03/07/2023

Larger language models do in-context learning differently

We study how in-context learning (ICL) in language models is affected by...

0 Jerry Wei, et al. ∙

research

∙ 02/06/2023

Data Selection for Language Models via Importance Resampling

Selecting a suitable training dataset is crucial for both general-domain...

0 Sang Michael Xie, et al. ∙

research

∙ 11/28/2022

What learning algorithm is in-context learning? Investigations with linear models

Neural sequence models, especially transformers, exhibit a remarkable ca...

0 Ekin Akyürek, et al. ∙

research

∙ 11/27/2022

A Theoretical Study of Inductive Biases in Contrastive Learning

Understanding self-supervised learning is important but challenging. Pre...

0 Jeff Z. HaoChen, et al. ∙

research

∙ 11/21/2022

First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains

Real-world machine learning applications often involve deploying neural ...

0 Kefan Dong, et al. ∙

research

∙ 11/10/2022

How Does Sharpness-Aware Minimization Minimize Sharpness?

Sharpness-Aware Minimization (SAM) is a highly effective regularization ...

0 Kaiyue Wen, et al. ∙

research

∙ 10/25/2022

Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models

Language modeling on large-scale datasets leads to impressive performanc...

0 Hong Liu, et al. ∙

research

∙ 07/18/2022

Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift

We often see undesirable tradeoffs in robust machine learning where out-...

0 Ananya Kumar, et al. ∙

research

∙ 06/16/2022

Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence

A major challenge in modern machine learning is theoretically understand...

0 Margalit Glasgow, et al. ∙

research

∙ 06/06/2022

Asymptotic Instance-Optimal Algorithms for Interactive Decision Making

Past research on interactive decision making problems (bandits, reinforc...

0 Kefan Dong, et al. ∙

research

∙ 05/22/2022

Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path

We revisit the incremental autonomous exploration problem proposed by Li...

0 Haoyuan Cai, et al. ∙

research

∙ 04/21/2022

Toward Fast, Flexible, and Robust Low-Light Image Enhancement

Existing low-light image enhancement techniques are mostly not only diff...

17 Long Ma, et al. ∙

research

∙ 04/06/2022

Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations

Contrastive learning is a highly effective method which uses unlabeled d...

0 Jeff Z. HaoChen, et al. ∙

research

∙ 04/01/2022

Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

We consider unsupervised domain adaptation (UDA), where labeled data fro...

6 Kendrick Shen, et al. ∙

research

∙ 02/21/2022

Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

When transferring a pretrained model to a downstream task, two popular m...

0 Ananya Kumar, et al. ∙

research

∙ 02/15/2022

Safe Reinforcement Learning by Imagining the Near Future

Safe reinforcement learning is a promising path toward applying reinforc...

0 Garrett Thomas, et al. ∙

research

∙ 12/09/2021

Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision

Images captured from low-light scenes often suffer from severe degradati...

0 Risheng Liu, et al. ∙

research

∙ 12/09/2021

DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization

Despite overparameterization, deep networks trained via supervised learn...

0 Aviral Kumar, et al. ∙

research

∙ 11/22/2021

Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification

The idea of conservatism has led to significant progress in offline rein...

0 Ling Pan, et al. ∙

research

∙ 11/03/2021

An Explanation of In-context Learning as Implicit Bayesian Inference

Large pretrained language models such as GPT-3 have the surprising abili...

7 Sang Michael Xie, et al. ∙

research

∙ 10/11/2021

Self-supervised Learning is More Robust to Dataset Imbalance

Self-supervised learning (SSL) is a scalable way to learn general visual...

4 Hong Liu, et al. ∙

research

∙ 08/04/2021

Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations

Training-time safety violations have been a major concern when we deploy...

0 Yuping Luo, et al. ∙

research

∙ 07/28/2021

Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers

A common lens to theoretically study neural net architectures is to anal...

0 Colin Wei, et al. ∙

research

∙ 07/12/2021

Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration

When facing uncertainty, decision-makers want predictions they can trust...

0 Shengjia Zhao, et al. ∙

research

∙ 06/18/2021

Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments

Domain generalization aims at performing well on unseen test environment...

0 Yining Chen, et al. ∙

research

∙ 06/17/2021

Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning

Pretrained language models have achieved state-of-the-art performance wh...

9 Colin Wei, et al. ∙

research

∙ 06/11/2021

Label Noise SGD Provably Prefers Flat Global Minimizers

In overparametrized models, the noise in stochastic gradient descent (SG...

9 Alex Damian, et al. ∙

research

∙ 06/09/2021

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

Recent work (Takanobu et al., 2020) proposed the system-wise evaluation ...

0 Zichuan Lin, et al. ∙

research

∙ 06/08/2021

Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

Recent works in self-supervised learning have advanced the state-of-the-...

2 Jeff Z. HaoChen, et al. ∙

research

∙ 03/24/2021

Why Do Local Methods Solve Nonconvex Problems?

Non-convex optimization is ubiquitous in modern machine learning. Resear...

0 Tengyu Ma, et al. ∙

research

∙ 02/09/2021

Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap

This paper presents a new model-free algorithm for episodic finite-horiz...

0 Haike Xu, et al. ∙

research

∙ 02/08/2021

Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature

This paper studies model-based bandit and reinforcement learning (RL) wi...

0 Kefan Dong, et al. ∙

research

∙ 12/08/2020

In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness

Consider a prediction setting where a few inputs (e.g., satellite images...

8 Sang Michael Xie, et al. ∙

research

∙ 11/03/2020

Meta-learning Transferable Representations with a Single Target Domain

Recent works found that fine-tuning and joint training—two popular appro...

0 Hong Liu, et al. ∙

research

∙ 10/22/2020

Beyond Lazy Training for Over-parameterized Tensor Decomposition

Over-parametrization is an important technique in training neural networ...

0 Xiang Wang, et al. ∙

research

∙ 10/21/2020

Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling

Document-level relation extraction (RE) poses new challenges compared to...

0 Wenxuan Zhou, et al. ∙

research

∙ 10/07/2020

Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data

Self-training algorithms, which train a model to fit pseudolabels predic...

18 Colin Wei, et al. ∙

research

∙ 08/27/2020

Entity and Evidence Guided Relation Extraction for DocRED

Document-level relation extraction is a challenging task which requires ...

0 Kevin Huang, et al. ∙

research

∙ 07/09/2020

Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK

We consider the dynamic of gradient descent for learning a two-layer neu...

8 Yuanzhi Li, et al. ∙

research

∙ 06/29/2020

Simplifying Models with Unlabeled Output Data

We focus on prediction problems with high-dimensional outputs that are s...

8 Sang Michael Xie, et al. ∙

Tengyu Ma

Featured Co-authors