b'Zhiyuan Li'

research

∙ 08/12/2023

Distributionally Robust Optimization and Invariant Representation Learning for Addressing Subgroup Underrepresentation: Mechanisms and Limitations

Spurious correlation caused by subgroup underrepresentation has received...

0 Nilesh Kumar, et al. ∙

research

∙ 07/27/2023

The Marginal Value of Momentum for Small Learning Rate SGD

Momentum is known to accelerate the convergence of gradient descent in s...

0 Runzhe Wang, et al. ∙

research

∙ 07/27/2023

Exploring Annotation-free Image Captioning with Retrieval-augmented Pseudo Sentence Generation

Training an image captioner without annotated image-sentence pairs has g...

0 Zhiyuan Li, et al. ∙

research

∙ 07/22/2023

Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation

For multi-modal magnetic resonance (MR) brain tumor image segmentation, ...

0 Yafei Zhang, et al. ∙

research

∙ 07/20/2023

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization

Despite extensive studies, the underlying reason as to why overparameter...

0 Kaiyue Wen, et al. ∙

research

∙ 06/22/2023

The Inductive Bias of Flatness Regularization for Deep Matrix Factorization

Recent works on over-parameterized neural networks have shown that the s...

0 Khashayar Gatmiry, et al. ∙

research

∙ 05/23/2023

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Given the massive cost of language model pre-training, a non-trivial imp...

0 Hong Liu, et al. ∙

research

∙ 04/16/2023

Learning Empirical Bregman Divergence for Uncertain Distance Representation

Deep metric learning techniques have been used for visual representation...

0 Zhiyuan Li, et al. ∙

research

∙ 02/20/2023

A Novel Collaborative Self-Supervised Learning Method for Radiomic Data

The computer-aided disease diagnosis from radiomic data is important in ...

0 Zhiyuan Li, et al. ∙

research

∙ 01/31/2023

Learning Generalized Hybrid Proximity Representation for Image Recognition

Recently, deep metric learning techniques received attention, as the lea...

0 Zhiyuan Li, et al. ∙

research

∙ 01/27/2023

Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing

It is believed that Gradient Descent (GD) induces an implicit bias towar...

0 Jikai Jin, et al. ∙

research

∙ 11/10/2022

How Does Sharpness-Aware Minimization Minimize Sharpness?

Sharpness-Aware Minimization (SAM) is a highly effective regularization ...

0 Kaiyue Wen, et al. ∙

research

∙ 11/02/2022

Interpretable Modeling and Reduction of Unknown Errors in Mechanistic Operators

Prior knowledge about the imaging physics provides a mechanistic forward...

6 Maryam Toloubidokhti, et al. ∙

research

∙ 10/25/2022

Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models

Language modeling on large-scale datasets leads to impressive performanc...

0 Hong Liu, et al. ∙

research

∙ 10/06/2022

Few-shot Generation of Personalized Neural Surrogates for Cardiac Simulation via Bayesian Meta-Learning

Clinical adoption of personalized virtual heart simulations faces challe...

0 Xiajun Jiang, et al. ∙

research

∙ 07/08/2022

Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent

As part of the effort to understand implicit bias of gradient descent in...

8 Zhiyuan Li, et al. ∙

research

∙ 06/14/2022

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

Normalization layers (e.g., Batch Normalization, Layer Normalization) we...

0 Kaifeng Lyu, et al. ∙

research

∙ 05/19/2022

Understanding Gradient Descent on Edge of Stability in Deep Learning

Deep learning experiments in Cohen et al. (2021) using deterministic Gra...

194 Sanjeev Arora, et al. ∙

research

∙ 05/16/2022

Peridynamic modeling for impact failure of wet concrete considering the influence of saturation

In this paper, a modified intermediately homogenized peridynamic (IH-PD)...

0 Liwei Wu, et al. ∙

research

∙ 03/24/2022

Interpretability of Neural Network With Physiological Mechanisms

Deep learning continues to play as a powerful state-of-art technique tha...

0 Anna Zou, et al. ∙

research

∙ 02/08/2022

A Novel Ontology-guided Attribute Partitioning Ensemble Learning Model for Early Prediction of Cognitive Deficits using Quantitative Structural MRI in Very Preterm Infants

Structural magnetic resonance imaging studies have shown that brain anat...

0 Zhiyuan Li, et al. ∙

research

∙ 02/02/2022

Robust Training of Neural Networks using Scale Invariant Architectures

In contrast to SGD, adaptive gradient methods like Adam allow robust tra...

6 Zhiyuan Li, et al. ∙

research

∙ 11/25/2021

An inverse problem of determining fractional orders in a fractal solute transport model

A fractal mobile-immobile (MIM in short) solute transport model in porou...

0 Gongsheng Li, et al. ∙

research

∙ 10/26/2021

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias

The generalization mystery of overparametrized deep nets has motivated e...

6 Kaifeng Lyu, et al. ∙

research

∙ 10/13/2021

What Happens after SGD Reaches Zero Loss? –A Mathematical Framework

Understanding the implicit bias of Stochastic Gradient Descent (SGD) is ...

30 Zhiyuan Li, et al. ∙

research

∙ 08/20/2021

DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection

Few-shot object detection, which aims at detecting novel objects rapidly...

0 Limeng Qiao, et al. ∙

research

∙ 08/18/2021

A reduced model for compressible viscous heat-conducting multicomponent flows

In the present paper we propose a reduced temperature non-equilibrium mo...

0 Chao Zhang, et al. ∙

research

∙ 03/25/2021

Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning

This paper considers batch Reinforcement Learning (RL) with general valu...

1 Yaqi Duan, et al. ∙

research

∙ 03/24/2021

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors

When fitting statistical models, some predictors are often found to be c...

0 Zhiyuan Li, et al. ∙

research

∙ 02/24/2021

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)

It is generally recognized that finite learning rate (LR), in contrast t...

12 Zhiyuan Li, et al. ∙

research

∙ 12/17/2020

Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning

Matrix factorization is a simple and natural test-bed to investigate the...

0 Zhiyuan Li, et al. ∙

research

∙ 10/16/2020

Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets?

Convolutional neural networks often dominate fully-connected counterpart...

25 Zhiyuan Li, et al. ∙

research

∙ 10/06/2020

Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate

Recent works (e.g., (Li and Arora, 2020)) suggest that the use of popula...

0 Zhiyuan Li, et al. ∙

research

∙ 07/18/2020

Learning Geometry-Dependent and Physics-Based Inverse Image Reconstruction

Deep neural networks have shown great potential in image reconstruction ...

0 Xiajun Jiang, et al. ∙

research

∙ 06/10/2020

When is Particle Filtering Efficient for POMDP Sequential Planning?

Particle filtering is a popular method for inferring latent states in st...

1 Simon S. Du, et al. ∙

research

∙ 05/22/2020

Semi-supervised Medical Image Classification with Global Latent Mixing

Computer-aided diagnosis via deep learning relies on large-scale annotat...

0 Prashnna Kumar Gyawali, et al. ∙

research

∙ 02/24/2020

Progressive Learning and Disentanglement of Hierarchical Representations

Learning rich representation from data is an important task for deep gen...

0 Zhiyuan Li, et al. ∙

research

∙ 11/18/2019

Implicit Regularization of Normalization Methods

Normalization methods such as batch normalization are commonly used in o...

17 Xiaoxia Wu, et al. ∙

research

∙ 11/03/2019

Enhanced Convolutional Neural Tangent Kernels

Recent research shows that for training with ℓ_2 loss, convolutional neu...

17 Zhiyuan Li, et al. ∙

research

∙ 10/16/2019

An Exponential Learning Rate Schedule for Deep Learning

Intriguing empirical evidence exists that deep learning can work well wi...

36 Zhiyuan Li, et al. ∙

research

∙ 10/03/2019

Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks

Recent research shows that the following two models are equivalent: (a) ...

16 Sanjeev Arora, et al. ∙

research

∙ 09/03/2019

Improving Disentangled Representation Learning with the Beta Bernoulli Process

To improve the ability of VAE to disentangle in the latent space, existi...

0 Prashnna Kumar Gyawali, et al. ∙

research

∙ 07/22/2019

Semi-Supervised Learning by Disentangling and Self-Ensembling Over Stochastic Latent Space

The success of deep learning in medical imaging is mostly achieved at th...

0 Prashnna Kumar Gyawali, et al. ∙

research

∙ 06/14/2019

Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets

Mode connectivity is a surprising phenomenon in the loss landscape of de...

3 Rohith Kuditipudi, et al. ∙

research

∙ 06/06/2019

Feature-level and Model-level Audiovisual Fusion for Emotion Recognition in the Wild

Emotion recognition plays an important role in human-computer interactio...

0 Jie Cai, et al. ∙

research

∙ 05/27/2019

Understanding Generalization of Deep Neural Networks Trained with Noisy Labels

Over-parameterized deep neural networks trained by simple first-order me...

0 Wei Hu, et al. ∙

research

∙ 04/26/2019

On Exact Computation with an Infinitely Wide Neural Net

How well does a classic deep net architecture like AlexNet or VGG19 clas...

0 Sanjeev Arora, et al. ∙

research

∙ 03/19/2019

Identity-Free Facial Expression Recognition using conditional Generative Adversarial Network

In this paper, we proposed a novel Identity-free conditional Generative ...

0 Jie Cai, et al. ∙

research

∙ 01/24/2019

Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

Recent works have cast some light on the mystery of why deep nets fit an...

16 Sanjeev Arora, et al. ∙

research

∙ 12/17/2018

Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition

In this paper, we proposed a novel Probabilistic Attribute Tree-CNN (PAT...

0 Jie Cai, et al. ∙

Zhiyuan Li

Featured Co-authors