research
∙
09/07/2023
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Despite their impressive capabilities, large language models (LLMs) are ...
share
research
∙
09/06/2023
Deep Reinforcement Learning from Hierarchical Weak Preference Feedback
Reward design is a fundamental, yet challenging aspect of practical rein...
research
∙
08/17/2023
Do you really follow me? Adversarial Instructions for Evaluating the Robustness of Large Language Models
Large Language Models (LLMs) have shown remarkable proficiency in follow...
research
∙
07/28/2023
Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system
Meetings play a critical infrastructural role in the coordination of wor...
research
∙
06/20/2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Transformer models have achieved remarkable results in various natural l...
research
∙
06/05/2023
Interactive Editing for Text Summarization
Summarizing lengthy documents is a common and essential task in our dail...
research
∙
05/23/2023
Query Rewriting for Retrieval-Augmented Large Language Models
Large Language Models (LLMs) play a powerful Reader of the Retrieve-then...
research
∙
05/11/2023
PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
Based on the remarkable achievements of pre-trained language models in a...
research
∙
05/09/2023
Summarization with Precise Length Control
Many applications of text generation such as summarization benefit from ...
research
∙
05/04/2023
ChatGPT-steered Editing Instructor for Customization of Abstractive Summarization
Tailoring outputs of large language models, such as ChatGPT, to specific...
research
∙
05/01/2023
In-Context Learning Unlocked for Diffusion Models
We present Prompt Diffusion, a framework for enabling in-context learnin...
research
∙
04/29/2023
POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models
Through prompting, large-scale pre-trained models have become more expre...
research
∙
04/25/2023
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Diffusion models are powerful, but they require a lot of time and data t...
research
∙
04/06/2023
Instruction Tuning with GPT-4
Prior work has shown that finetuning large language models (LLMs) using ...
research
∙
03/18/2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Fine-tuning large pre-trained language models on downstream tasks has be...
research
∙
02/24/2023
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Large language models (LLMs), such as ChatGPT, are able to generate huma...
research
∙
02/22/2023
Guiding Large Language Models via Directional Stimulus Prompting
We introduce a new framework, Directional Stimulus Prompting, that uses ...
research
∙
02/08/2023
A Prototype-Oriented Clustering for Domain Shift with Source Privacy
Unsupervised clustering under domain shift (UCDS) studies how to transfe...
research
∙
12/20/2022
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization
Dialogue summarization has recently garnered significant attention due t...
research
∙
12/08/2022
Momentum Calibration for Text Generation
The input and output of most text generation tasks can be transformed to...
research
∙
11/22/2022
HyperTuning: Toward Adapting Large Language Models without Back-propagation
Fine-tuning large language models for different tasks can be costly and ...
research
∙
10/04/2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Layer-wise distillation is a powerful tool to compress large models (i.e...
research
∙
07/08/2022
OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering
The information in tables can be an important complement to text, making...
research
∙
06/25/2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in va...
research
∙
06/22/2022
GODEL: Large-Scale Pre-Training for Goal-Directed Dialog
We introduce GODEL (Grounded Open Dialogue Language Model), a large pre-...
research
∙
06/05/2022
Diffusion-GAN: Training GANs with Diffusion
For stable training of generative adversarial networks (GANs), injecting...
research
∙
05/10/2022
ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Active learning, which effectively collects informative unlabeled data f...
research
∙
04/15/2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Pre-trained language models have demonstrated superior performance in va...
research
∙
04/13/2022
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing
Model ensemble is a popular approach to produce a low-variance and well-...
research
∙
02/19/2022
Truncated Diffusion Probabilistic Models
Employing a forward Markov diffusion chain to gradually map the data to ...
research
∙
02/14/2022
Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs
Token-mixing multi-layer perceptron (MLP) models have shown competitive ...
research
∙
02/06/2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models
Recent research has shown the existence of significant redundancy in lar...
research
∙
12/06/2021
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
Most of today's AI systems focus on using self-attention mechanisms and ...
research
∙
11/18/2021
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
This paper presents a new pre-trained language model, DeBERTaV3, which i...
research
∙
09/15/2021
ARCH: Efficient Adversarial Regularized Training with Caching
Adversarial regularization can improve model generalization in many natu...
research
∙
05/25/2021
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization
The Lottery Ticket Hypothesis suggests that an over-parametrized network...
research
∙
04/11/2021
Adversarial Training as Stackelberg Game: An Unrolled Optimization Approach
Adversarial training has been shown to improve the generalization perfor...
research
∙
03/20/2021
Token-wise Curriculum Learning for Neural Machine Translation
Existing curriculum learning approaches to Neural Machine Translation (N...
research
∙
02/23/2021
Greedy Multi-step Off-Policy Reinforcement Learning
Multi-step off-policy reinforcement learning has achieved great success....
research
∙
01/01/2021
Reader-Guided Passage Reranking for Open-Domain Question Answering
Current open-domain question answering (QA) systems often follow a Retri...
research
∙
01/01/2021
UnitedQA: A Hybrid Approach for Open Domain Question Answering
To date, most of recent work under the retrieval-reader framework for op...
research
∙
01/01/2021
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020. The competition...
research
∙
09/17/2020
Generation-Augmented Retrieval for Open-domain Question Answering
Conventional sparse retrieval methods such as TF-IDF and BM25 are simple...
research
∙
06/05/2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
Recent progress in pre-trained neural language models has significantly ...
research
∙
04/29/2020
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning
In this work, we aim at equipping pre-trained language models with struc...
research
∙
04/20/2020
Adversarial Training for Large Neural Language Models
Generalization and robustness are both key desiderata for designing mach...
research
∙
02/19/2020
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
We present MT-DNN, an open-source natural language understanding (NLU) t...
research
∙
11/08/2019
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
Transfer learning has fundamentally changed the landscape of natural lan...
research
∙
08/21/2019
X-SQL: reinforce schema representation with context
In this work, we present X-SQL, a new network architecture for the probl...
research
∙
08/08/2019