Simiao Zuo | DeepAI

DeepAI AI Chat

Featured Co-authors

Jianfeng Gao
234 publications
Chao Zhang
211 publications
Hongyuan Zha
95 publications
Tuo Zhao
88 publications
Weizhu Chen
87 publications
Yue Yu
87 publications
Xiaodong Liu
86 publications
Chen Liang
53 publications
Pengcheng He
47 publications
Hongteng Xu
41 publications
Haoming Jiang
32 publications

∙ 12/15/2022

Efficient Long Sequence Modeling via State Space Augmented Transformer

Transformer models have achieved superior performance in various natural...

0 Simiao Zuo, et al. ∙

∙ 10/04/2022

Less is More: Task-aware Layer-wise Distillation for Language Model Compression

Layer-wise distillation is a powerful tool to compress large models (i.e...

0 Chen Liang, et al. ∙

∙ 09/15/2022

Context-Aware Query Rewriting for Improving Users' Search Experience on E-commerce Websites

E-commerce queries are often short and ambiguous. Consequently, query un...

0 Simiao Zuo, et al. ∙

∙ 09/15/2022

DiP-GNN: Discriminative Pre-Training of Graph Neural Networks

Graph neural network (GNN) pre-training methods have been proposed to en...

0 Simiao Zuo, et al. ∙

∙ 09/15/2022

Differentially Private Estimation of Hawkes Process

Point process models are of great importance in real world applications....

0 Simiao Zuo, et al. ∙

∙ 06/25/2022

PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance

Large Transformer-based models have exhibited superior performance in va...

0 Qingru Zhang, et al. ∙

∙ 04/15/2022

MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

Pre-trained language models have demonstrated superior performance in va...

0 Simiao Zuo, et al. ∙

∙ 02/06/2022

No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models

Recent research has shown the existence of significant redundancy in lar...

0 Chen Liang, et al. ∙

∙ 10/08/2021

Taming Sparsely Activated Transformer with Stochastic Experts

Sparsely activated models (SAMs), such as Mixture-of-Experts (MoE), can ...

2 Simiao Zuo, et al. ∙

∙ 09/15/2021

Self-Training with Differentiable Teacher

Self-training achieves enormous success in various semi-supervised and w...

0 Simiao Zuo, et al. ∙

∙ 09/15/2021

ARCH: Efficient Adversarial Regularized Training with Caching

Adversarial regularization can improve model generalization in many natu...

0 Simiao Zuo, et al. ∙

∙ 05/25/2021

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

The Lottery Ticket Hypothesis suggests that an over-parametrized network...

0 Chen Liang, et al. ∙

∙ 04/11/2021

Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach

Adversarial training has been shown to improve the generalization perfor...

0 Simiao Zuo, et al. ∙

∙ 11/30/2020

A Hypergradient Approach to Robust Regression without Correspondence

We consider a regression problem, where the correspondence between input...

0 Yujia Xie, et al. ∙

∙ 10/15/2020

Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach

Fine-tuned pre-trained language models (LMs) achieve enormous success in...

0 Yue Yu, et al. ∙

∙ 02/21/2020

Transformer Hawkes Process

Modern data acquisition routinely produce massive amounts of event seque...

5 Simiao Zuo, et al. ∙

∙ 12/02/2018

Image Score: How to Select Useful Samples

There has long been debates on how we could interpret neural networks an...

0 Simiao Zuo, et al. ∙