Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text

12/08/2020
by   Nishtha Madaan, et al.
0

Machine Learning has seen tremendous growth recently, which has led to a larger adoption of ML systems for educational assessments, credit risk, healthcare, employment, criminal justice, to name a few. Trustworthiness of ML and NLP systems is a crucial aspect and requires guarantee that the decisions they make are fair and robust. Aligned with this, we propose a framework GYC, to generate a set of counterfactual text samples, which are crucial for testing these ML systems. Our main contributions include a) We introduce GYC, a framework to generate counterfactual samples such that the generation is plausible, diverse, goal-oriented, and effective, b) We generate counterfactual samples, that can direct the generation towards a corresponding condition such as named-entity tag, semantic role label, or sentiment. Our experimental results on various domains show that GYC generates counterfactual text samples exhibiting the above four properties. be fed complementary to the existing data augmentation for improving the debiasing algorithms performance as compared to existing counterfactuals generated by token substitution. GYC generates counterfactuals that can act as test cases to evaluate a model and any text debiasing algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2022

Plug and Play Counterfactual Text Generation for Model Robustness

Generating counterfactual test-cases is an important backbone for testin...
research
06/06/2021

Empowering Language Understanding with Counterfactual Reasoning

Present language understanding methods have demonstrated extraordinary a...
research
05/26/2023

CREST: A Joint Framework for Rationalization and Counterfactual Text Generation

Selective rationales and counterfactual examples have emerged as two eff...
research
08/24/2020

PermuteAttack: Counterfactual Explanation of Machine Learning Credit Scorecards

This paper is a note on new directions and methodologies for validation ...
research
04/02/2021

Sketch and Customize: A Counterfactual Story Generator

Recent text generation models are easy to generate relevant and fluent t...
research
04/12/2021

Consequence-aware Sequential Counterfactual Generation

Counterfactuals have become a popular technique nowadays for interacting...
research
09/29/2014

The Utility of Text: The Case of Amicus Briefs and the Supreme Court

We explore the idea that authoring a piece of text is an act of maximizi...

Please sign up or login with your details

Forgot password? Click here to reset