research
∙
06/30/2023
DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries
Named entity recognition (NER) is a crucial task for online advertisemen...
share
research
∙
12/15/2022
Efficient Long Sequence Modeling via State Space Augmented Transformer
Transformer models have achieved superior performance in various natural...
research
∙
10/04/2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Layer-wise distillation is a powerful tool to compress large models (i.e...
research
∙
09/15/2022
Context-Aware Query Rewriting for Improving Users' Search Experience on E-commerce Websites
E-commerce queries are often short and ambiguous. Consequently, query un...
research
∙
09/15/2022
DiP-GNN: Discriminative Pre-Training of Graph Neural Networks
Graph neural network (GNN) pre-training methods have been proposed to en...
research
∙
09/15/2022
Differentially Private Estimation of Hawkes Process
Point process models are of great importance in real world applications....
research
∙
06/25/2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in va...
research
∙
04/15/2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Pre-trained language models have demonstrated superior performance in va...
research
∙
02/06/2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Recent research has shown the existence of significant redundancy in lar...
research
∙
10/08/2021
Taming Sparsely Activated Transformer with Stochastic Experts
Sparsely activated models (SAMs), such as Mixture-of-Experts (MoE), can ...
research
∙
09/15/2021
Self-Training with Differentiable Teacher
Self-training achieves enormous success in various semi-supervised and w...
research
∙
09/15/2021
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natu...
research
∙
05/25/2021
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network...
research
∙
04/11/2021
Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach
Adversarial training has been shown to improve the generalization perfor...
research
∙
11/30/2020
A Hypergradient Approach to Robust Regression without Correspondence
We consider a regression problem, where the correspondence between input...
research
∙
10/15/2020
Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach
Fine-tuned pre-trained language models (LMs) achieve enormous success in...
research
∙
02/21/2020
Transformer Hawkes Process
Modern data acquisition routinely produce massive amounts of event seque...
research
∙
12/02/2018