Efficient transfer learning for NLP with ELECTRA

04/06/2021

∙

Clark et al. [2020] claims that the ELECTRA approach is highly efficient in NLP performances relative to computation budget. As such, this reproducibility study focus on this claim, summarized by the following question: Can we use ELECTRA to achieve close to SOTA performances for NLP in low-resource settings, in term of compute cost?

READ FULL TEXT

Efficient transfer learning for NLP with ELECTRA

Reproducibility in NLP: What Have We Learned from the Checklist?

Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages

This joke is [MASK]: Recognizing Humor and Offense with Prompting

Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages

Learning to translate by learning to communicate

The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP

Lessons in Reproducibility: Insights from NLP Studies in Materials Science

Efficient transfer learning for NLP with ELECTRA

Related Research

Reproducibility in NLP: What Have We Learned from the Checklist?

Adapting to the Low-Resource Double-Bind: Investigating Low-Compute Methods on Low-Resource African Languages

This joke is [MASK]: Recognizing Humor and Offense with Prompting

Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages

Learning to translate by learning to communicate

The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP

Lessons in Reproducibility: Insights from NLP Studies in Materials Science