Human and Automatic Detection of Generated Text

by   Daphne Ippolito, et al.

With the advent of generative models with a billion parameters or more, it is now possible to automatically generate vast amounts of human-sounding text. This raises questions into just how human-like is the machine-generated text, and how long does a text excerpt need to be for both humans and automatic discriminators to be able reliably detect that it was machine-generated. In this paper, we conduct a thorough investigation of how choices such as sampling strategy and text excerpt length can impact the performance of automatic detection methods as well as human raters. We find that the sampling strategies which result in more human-like text according to human raters create distributional differences from human-written text that make detection easy for automatic discriminators.



page 1

page 2

page 3

page 4


Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions

Language generation models' democratization benefits many domains, from ...

Unsupervised and Distributional Detection of Machine-Generated Text

The power of natural language generation models has provoked a flurry of...

RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text

In recent years, large neural networks for natural language generation (...

Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies

Generating factual, long-form text such as Wikipedia articles raises thr...

The Creativity of Text-based Generative Art

Text-based generation of digital images has made a giant leap towards be...

Adversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers

The detection of computer-generated text is an area of rapidly increasin...

Creative Artificial Intelligence – Algorithms vs. humans in an incentivized writing competition

The release of openly available, robust text generation algorithms has s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.