Transformers: State-of-the-art Natural Language Processing

10/09/2019
by   Thomas Wolf, et al.
0

Recent advances in modern Natural Language Processing (NLP) research have been dominated by the combination of Transfer Learning methods with large-scale Transformer language models. With them came a paradigm shift in NLP with the starting point for training a model on a downstream task moving from a blank specific model to a general-purpose pretrained architecture. Still, creating these general-purpose models remains an expensive and time-consuming process restricting the use of these methods to a small sub-set of the wider NLP community. In this paper, we present Transformers, a library for state-of-the-art NLP, making these developments available to the community by gathering state-of-the-art general-purpose pretrained models under a unified API together with an ecosystem of libraries, examples, tutorials and scripts targeting many downstream NLP tasks. Transformers features carefully crafted model implementations and high-performance pretrained weights for two main deep learning frameworks, PyTorch and TensorFlow, while supporting all the necessary tools to analyze, evaluate and use these models in downstream tasks such as text/token classification, questions answering and language generation among others. Transformers has gained significant organic traction and adoption among both the researcher and practitioner communities. We are committed at Hugging Face to pursue the efforts to develop Transformers with the ambition of creating the standard library for building NLP systems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/09/2019

HuggingFace's Transformers: State-of-the-art Natural Language Processing

Recent advances in modern Natural Language Processing (NLP) research hav...
research
10/23/2020

Learning to Recognize Dialect Features

Linguists characterize dialects by the presence, absence, and frequency ...
research
09/19/2020

CLEVR Parser: A Graph Parser Library for Geometric Learning on Language Grounded Image Scenes

The CLEVR dataset has been used extensively in language grounded visual ...
research
08/03/2023

XNLP: An Interactive Demonstration System for Universal Structured NLP

Structured Natural Language Processing (XNLP) is an important subset of ...
research
08/01/2020

Trojaning Language Models for Fun and Profit

Recent years have witnessed a new paradigm of building natural language ...
research
10/02/2022

Wide Attention Is The Way Forward For Transformers

The Transformer is an extremely powerful and prominent deep learning arc...

Please sign up or login with your details

Forgot password? Click here to reset