EasyTransfer – A Simple and Scalable Deep Transfer Learning Platform for NLP Applications

11/18/2020
by   Minghui Qiu, et al.
63

The literature has witnessed the success of applying deep Transfer Learning (TL) algorithms to many NLP applications, yet it is not easy to build a simple and scalable TL toolkit for this purpose. To bridge this gap, the EasyTransfer platform is designed to make it easy to develop deep TL algorithms for NLP applications. It is built with rich API abstractions, a scalable architecture and comprehensive deep TL algorithms, to make the development of NLP applications easier. To be specific, the build-in data and model parallelism strategy shows to be 4x faster than the default distribution strategy of Tensorflow. EasyTransfer supports the mainstream pre-trained ModelZoo, including Pre-trained Language Models (PLMs) and multi-modality models. It also integrates various SOTA models for mainstream NLP applications in AppZoo, and supports mainstream TL algorithms as well. The toolkit is convenient for users to quickly start model training, evaluation, offline prediction, and online deployment. This system is currently deployed at Alibaba to support a variety of business scenarios, including item recommendation, personalized search, and conversational question answering. Extensive experiments on real-world datasets show that EasyTransfer is suitable for online production with cutting-edge performance. The source code of EasyTransfer is released at Github (https://github.com/alibaba/EasyTransfer).

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/03/2021

OpenPrompt: An Open-source Framework for Prompt-learning

Prompt-learning has become a new paradigm in modern natural language pro...
09/24/2020

N-LTP: A Open-source Neural Chinese Language Technology Platform with Pretrained Models

We introduce N-LTP, an open-source Python Chinese natural language proce...
11/20/2019

Transfer Learning Toolkit: Primers and Benchmarks

The transfer learning toolkit wraps the codes of 17 transfer learning mo...
01/15/2021

PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation

Image Segmentation plays an essential role in computer vision and image ...
07/15/2021

Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Mining

Many real-world applications involve the use of Optical Character Recogn...
07/15/2021

Multi-Task Learning based Online Dialogic Instruction Detection with Pre-trained Language Models

In this work, we study computational approaches to detect online dialogi...
08/12/2020

The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models

We present the Language Interpretability Tool (LIT), an open-source plat...

Code Repositories

EasyTransfer

EasyTransfer is designed to make the development of transfer learning in NLP applications easier.


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.