research
∙
07/15/2023
CPET: Effective Parameter-Efficient Tuning for Compressed Large Language Models
Parameter-efficient tuning (PET) has been widely explored in recent year...
share
research
∙
05/28/2023
Emergent Modularity in Pre-trained Transformers
This work examines the presence of modularity in pre-trained Transformer...
research
∙
05/28/2023
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Injecting external knowledge can improve the performance of pre-trained ...
research
∙
05/28/2023
Plug-and-Play Document Modules for Pre-trained Models
Large-scale pre-trained models (PTMs) have been widely used in document-...
research
∙
05/02/2023
UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language Models
Recent research demonstrates that external knowledge injection can advan...
research
∙
04/06/2023
Bridging the Language Gap: Knowledge Injected Multilingual Question Answering
Question Answering (QA) is the task of automatically answering questions...
research
∙
02/14/2023
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises
For many real-world applications, the user-generated inputs usually cont...
research
∙
11/14/2022
Finding Skill Neurons in Pre-trained Transformer-based Language Models
Transformer-based pre-trained language models have demonstrated superior...
research
∙
09/20/2022
Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models
Prompting, which casts downstream applications as language modeling task...
research
∙
07/12/2022
Effective Few-Shot Named Entity Linking by Meta-Learning
Entity linking aims to link ambiguous mentions to their corresponding en...
research
∙
06/19/2022
A Unified Understanding of Deep NLP Models for Text Classification
The rapid development of deep natural language processing (NLP) models f...
research
∙
05/23/2022
Prompt Tuning for Discriminative Pre-trained Language Models
Recent works have shown promising results of prompt tuning in stimulatin...
research
∙
10/05/2021
MoEfication: Conditional Computation of Transformer Models for Efficient Inference
Transformer-based pre-trained language models can achieve superior perfo...
research
∙
09/24/2021
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models
Pre-Trained Vision-Language Models (VL-PTMs) have shown promising capabi...
research
∙
06/01/2021
SHUOWEN-JIEZI: Linguistically Informed Tokenizers For Chinese Language Model Pretraining
Conventional tokenization methods for Chinese pretrained language models...
research
∙
05/28/2021
Knowledge Inheritance for Pre-trained Language Models
Recent explorations of large-scale pre-trained language models (PLMs) su...
research
∙
05/26/2021
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Backdoor attacks are a kind of insidious security threat against machine...
research
∙
02/07/2021
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models
Fine-tuning pre-trained language models (PLMs) has demonstrated its effe...
research
∙
01/18/2021
Red Alarm for Pre-trained Models: Universal Vulnerabilities by Neuron-Level Backdoor Attacks
Due to the success of pre-trained models (PTMs), people usually fine-tun...
research
∙
12/01/2020
CPM: A Large-scale Generative Chinese Pre-trained Language Model
Pre-trained Language Models (PLMs) have proven to be beneficial for vari...
research
∙
09/29/2020
Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
Several recent efforts have been devoted to enhancing pre-trained langua...
research
∙
04/21/2020
Train No Evil: Selective Masking for Task-guided Pre-training
Recently, pre-trained language models mostly follow the pre-training-the...
research
∙
11/05/2019
Adversarial Language Games for Advanced Natural Language Intelligence
While adversarial games have been well studied in various board games an...
research
∙
05/17/2019
ERNIE: Enhanced Language Representation with Informative Entities
Neural language representation models such as BERT pre-trained on large-...
research
∙
12/21/2018
COSINE: Compressive Network Embedding on Large-scale Information Networks
There is recently a surge in approaches that learn low-dimensional embed...
research
∙
12/20/2018