research
∙
09/18/2023
Pivotal Estimation of Linear Discriminant Analysis in High Dimensions
We consider the linear discriminant analysis problem in the high-dimensi...
share
research
∙
09/06/2023
Deep Reinforcement Learning from Hierarchical Weak Preference Feedback
Reward design is a fundamental, yet challenging aspect of practical rein...
research
∙
07/24/2023
Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems
A crucial task in decision-making problems is reward engineering. It is ...
research
∙
07/04/2023
Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks
Convolutional residual neural networks (ConvResNets), though overparamet...
research
∙
06/26/2023
Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories
Existing theories on deep nonparametric regression have shown that when ...
research
∙
06/20/2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Transformer models have achieved remarkable results in various natural l...
research
∙
03/18/2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Fine-tuning large pre-trained language models on downstream tasks has be...
research
∙
02/25/2023
On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds
Generative networks have experienced great empirical successes in distri...
research
∙
02/19/2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers
Knowledge distillation has been shown to be a powerful model compression...
research
∙
02/14/2023
Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
Diffusion models achieve state-of-the-art performance in various generat...
research
∙
12/15/2022
Efficient Long Sequence Modeling via State Space Augmented Transformer
Transformer models have achieved superior performance in various natural...
research
∙
12/01/2022
High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization
Label Shift has been widely believed to be harmful to the generalization...
research
∙
10/04/2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Layer-wise distillation is a powerful tool to compress large models (i.e...
research
∙
09/21/2022
First-order Policy Optimization for Robust Markov Decision Process
We consider the problem of solving robust Markov decision process (MDP),...
research
∙
09/15/2022
Context-Aware Query Rewriting for Improving Users' Search Experience on E-commerce Websites
E-commerce queries are often short and ambiguous. Consequently, query un...
research
∙
09/15/2022
DiP-GNN: Discriminative Pre-Training of Graph Neural Networks
Graph neural network (GNN) pre-training methods have been proposed to en...
research
∙
09/15/2022
Differentially Private Estimation of Hawkes Process
Point process models are of great importance in real world applications....
research
∙
06/25/2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in va...
research
∙
06/06/2022
Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networks
We consider the off-policy evaluation problem of reinforcement learning ...
research
∙
05/04/2022
A Manifold Two-Sample Test Study: Integral Probability Metric with Neural Networks
Two-sample tests are important areas aiming to determine whether two col...
research
∙
04/15/2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Pre-trained language models have demonstrated superior performance in va...
research
∙
04/13/2022
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Model ensemble is a popular approach to produce a low-variance and well-...
research
∙
04/08/2022
CERES: Pretraining of Graph-Conditioned Transformer for Semi-Structured Session Data
User sessions empower many search and recommendation tasks on a daily ba...
research
∙
02/07/2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably
We investigate the role of noise in optimization algorithms for learning...
research
∙
02/06/2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Recent research has shown the existence of significant redundancy in lar...
research
∙
01/24/2022
Homotopic Policy Mirror Descent: Policy Convergence, Implicit Regularization, and Improved Sample Complexity
We propose the homotopic policy mirror descent (HPMD) method for solving...
research
∙
01/15/2022
Block Policy Mirror Descent
In this paper, we present a new class of policy gradient (PG) methods, n...
research
∙
01/06/2022
Deep Learning Assisted End-to-End Synthesis of mm-Wave Passive Networks with 3D EM Structures: A Study on A Transformer-Based Matching Network
This paper presents a deep learning assisted synthesis approach for dire...
research
∙
01/01/2022
Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces
Learning operators between infinitely dimensional spaces is an important...
research
∙
12/20/2021
Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning
Critical sectors of human society are progressing toward the adoption of...
research
∙
10/10/2021
Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits
Embedding learning has found widespread applications in recommendation s...
research
∙
10/08/2021
Taming Sparsely Activated Transformer with Stochastic Experts
Sparsely activated models (SAMs), such as Mixture-of-Experts (MoE), can ...
research
∙
10/07/2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect
Recent empirical advances show that training deep models with large lear...
research
∙
09/15/2021
Self-Training with Differentiable Teacher
Self-training achieves enormous success in various semi-supervised and w...
research
∙
09/15/2021
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natu...
research
∙
09/07/2021
Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks
Most of existing statistical theories on deep neural networks have sampl...
research
∙
08/19/2021
QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction
We study the problem of query attribute value extraction, which aims to ...
research
∙
08/15/2021
Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Data
Bregman proximal point algorithm (BPPA), as one of the centerpieces in t...
research
∙
06/16/2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
Weak supervision has shown promising results in many natural language pr...
research
∙
05/25/2021
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network...
research
∙
05/18/2021
Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approach
Multi-agent reinforcement learning (MARL) becomes more challenging in th...
research
∙
05/03/2021
COUnty aggRegation mixup AuGmEntation (COURAGE) COVID-19 Prediction
The global spread of COVID-19, the disease caused by the novel coronavir...
research
∙
04/11/2021
Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach
Adversarial training has been shown to improve the generalization perfor...
research
∙
03/20/2021
Token-wise Curriculum Learning for Neural Machine Translation
Existing curriculum learning approaches to Neural Machine Translation (N...
research
∙
03/01/2021
Reinforcement Learning for Adaptive Mesh Refinement
Large-scale finite element simulations of complex physical systems gover...
research
∙
02/24/2021
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization
Numerous empirical evidences have corroborated the importance of noise i...
research
∙
02/20/2021
Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach
Reliable automatic evaluation of dialogue systems under an interactive e...
research
∙
11/30/2020
A Hypergradient Approach to Robust Regression without Correspondence
We consider a regression problem, where the correspondence between input...
research
∙
11/03/2020
Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networks
Causal inference explores the causation between actions and the conseque...
research
∙
10/22/2020