MGR: Multi-generator based Rationalization

05/08/2023
by   Wei Liu, et al.
0

Rationalization is to employ a generator and a predictor to construct a self-explaining NLP model in which the generator selects a subset of human-intelligible pieces of the input text to the following predictor. However, rationalization suffers from two key challenges, i.e., spurious correlation and degeneration, where the predictor overfits the spurious or meaningless pieces solely selected by the not-yet well-trained generator and in turn deteriorates the generator. Although many studies have been proposed to address the two challenges, they are usually designed separately and do not take both of them into account. In this paper, we propose a simple yet effective method named MGR to simultaneously solve the two problems. The key idea of MGR is to employ multiple generators such that the occurrence stability of real pieces is improved and more meaningful pieces are delivered to the predictor. Empirically, we show that MGR improves the F1 score by up to 20.9 as compared to state-of-the-art methods. Codes are available at https://github.com/jugechengzi/Rationalization-MGR .

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/17/2022

FR: Folded Rationalization with a Unified Encoder

Conventional works generally employ a two-phase model in which a generat...
research
05/23/2023

Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipshitz Restraint

A self-explaining rationalization model is generally constructed by a co...
research
07/25/2020

Bollyrics: Automatic Lyrics Generator for Romanised Hindi

Song lyrics convey a meaningful story in a creative manner with complex ...
research
10/29/2019

Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control

Selective rationalization has become a common mechanism to ensure that p...
research
06/05/2023

Bootstrapped Training of Score-Conditioned Generator for Offline Design of Biological Sequences

We study the problem of optimizing biological sequences, e.g., proteins,...
research
10/29/2022

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

This work proposes a method for using any generator network as the found...
research
05/27/2023

Unsupervised Selective Rationalization with Noise Injection

A major issue with using deep learning models in sensitive applications ...

Please sign up or login with your details

Forgot password? Click here to reset