DeepAI
Log In Sign Up

AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization

03/21/2022
by   Moussa Kamal Eddine, et al.
0

Like most natural language understanding and generation tasks, state-of-the-art models for summarization are transformer-based sequence-to-sequence architectures that are pretrained on large corpora. While most existing models focused on English, Arabic remained understudied. In this paper we propose AraBART, the first Arabic model in which the encoder and the decoder are pretrained end-to-end, based on BART. We show that AraBART achieves the best performance on multiple abstractive summarization datasets, outperforming strong baselines including a pretrained Arabic BERT-based model and multilingual mBART and mT5 models.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/23/2020

BARThez: a Skilled Pretrained French Sequence-to-Sequence Model

Inductive transfer learning, enabled by self-supervised learning, have t...
06/01/2019

Efficient Adaptation of Pretrained Transformers for Abstractive Summarization

Large-scale learning of transformer language models has yielded improvem...
03/02/2022

Controlling the Focus of Pretrained Language Generation Models

The finetuning of pretrained transformer-based language generation model...
12/31/2020

AraGPT2: Pre-Trained Transformer for Arabic Language Generation

Recently, pretrained transformer-based architectures have proven to be v...
03/14/2020

Document Ranking with a Pretrained Sequence-to-Sequence Model

This work proposes a novel adaptation of a pretrained sequence-to-sequen...
06/01/2021

Comparing Test Sets with Item Response Theory

Recent years have seen numerous NLP datasets introduced to evaluate the ...
09/16/2021

A Bag of Tricks for Dialogue Summarization

Dialogue summarization comes with its own peculiar challenges as opposed...