research
∙
07/19/2023
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation
Rising computational demands of modern natural language processing (NLP)...
share
research
∙
05/24/2023
Large Language Model Distillation Doesn't Need a Teacher
Knowledge distillation trains a smaller student model to match the outpu...
research
∙
05/15/2023
TESS: Text-to-Text Self-Conditioned Simplex Diffusion
Diffusion models have emerged as a powerful paradigm for generation, obt...
research
∙
01/24/2023
The Semantic Scholar Open Data Platform
The volume of scientific output is creating an urgent need for automated...
research
∙
12/19/2022
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations
Although large language models can be prompted for both zero- and few-sh...
research
∙
10/27/2022
What Language Model to Train if You Have One Million GPU Hours?
The crystallization of modeling methods around the Transformer architect...
research
∙
10/25/2022
SciFact-Open: Towards open-domain scientific claim verification
While research on scientific claim verification has led to the developme...
research
∙
10/19/2022
Continued Pretraining for Better Zero- and Few-Shot Promptability
Recently introduced language model prompting methods can achieve high ac...
research
∙
10/14/2022
Transparency Helps Reveal When Language Models Learn Meaning
Many current NLP systems are built from language models trained to optim...
research
∙
04/12/2022
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
Large pretrained Transformer language models have been shown to exhibit ...
research
∙
03/16/2022
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search
Abstractive summarization systems today produce fluent and relevant outp...
research
∙
03/11/2022
Staged Training for Transformer Language Models
The current standard approach to scaling transformer language models tra...
research
∙
12/02/2021
LongChecker: Improving scientific claim verification by modeling full-abstract context
We introduce the LongChecker system for scientific claim verification. G...
research
∙
11/16/2021
Few-Shot Self-Rationalization with Natural Language Prompts
Self-rationalization models that predict task labels and generate free-t...
research
∙
10/16/2021
PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization
Recently proposed pre-trained generation models achieve strong performan...
research
∙
07/15/2021
FLEX: Unifying Evaluation for Few-Shot NLP
Few-shot NLP research is highly active, yet conducted in disjoint resear...
research
∙
06/17/2021
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
Biomedical knowledge graphs (KGs) hold rich information on entities such...
research
∙
05/07/2021
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Readers of academic research papers often read with the goal of answerin...
research
∙
04/18/2021
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts
Determining coreference of concept mentions across multiple documents is...
research
∙
04/13/2021
MS2: Multi-Document Summarization of Medical Studies
To assess the effectiveness of any medical intervention, researchers mus...
research
∙
01/02/2021
Cross-Document Language Modeling
We introduce a new pretraining approach for language models that are gea...
research
∙
05/01/2020
SciREX: A Challenge Dataset for Document-Level Information Extraction
Extracting information from full documents is an important problem in ma...
research
∙
04/23/2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
Language models pretrained on text from a wide variety of sources form t...
research
∙
04/15/2020
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Representation learning is a critical ingredient for natural language pr...
research
∙
04/15/2020
Document-level Representation Learning using Citation-informed Transformers
Representation learning is a critical ingredient for natural language pr...
research
∙
04/10/2020
Longformer: The Long-Document Transformer
Transformer-based models are unable to process long sequences due to the...
research
∙
09/09/2019
Pretrained Language Models for Sequential Sentence Classification
As a step toward better document-level understanding, we explore classif...
research
∙
03/26/2019
SciBERT: Pretrained Contextualized Embeddings for Scientific Text
Obtaining large-scale annotated data for NLP tasks in the scientific dom...
research
∙
02/20/2019
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
Despite recent advances in natural language processing, many statistical...
research
∙
10/30/2018
Improving Distant Supervision with Maxpooled Attention and Sentence-Level Supervision
We propose an effective multitask learning setup for reducing distant su...
research
∙
07/07/2018
Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff
The sensitivity of millimeter wave (mmWave) signals to blockages is a fu...
research
∙
05/06/2018