Show and Write: Entity-aware News Generation with Image Information

by   Zhongping Zhang, et al.

Automatically writing long articles is a complex and challenging language generation task. Prior work has primarily focused on generating these articles using human-written prompt to provide some topical context and some metadata about the article. That said, for many applications, such as generating news stories, these articles are often paired with images and their captions or alt-text, which in turn are based on real-world events and may reference many different named entities that are difficult to be correctly recognized and predicted by language models. To address these two problems, this paper introduces an Entity-aware News Generation method with Image iNformation, Engin, to incorporate news image information into language models. Engin produces news articles conditioned on both metadata and information such as captions and named entities extracted from images. We also propose an Entity-aware mechanism to help our model better recognize and predict the entity names in news. We perform experiments on two public large-scale news datasets, GoodNews and VisualNews. Quantitative results show that our approach improves article perplexity by 4-5 points over the base models. Qualitative results demonstrate the text generated by Engin is more consistent with news images. We also perform article quality annotation experiment on the generated articles to validate that our model produces higher-quality articles. Finally, we investigate the effect Engin has on methods that automatically detect machine-generated articles.


page 2

page 12

page 13

page 14

page 16

page 18

page 19


VisualNews : Benchmark and Challenges in Entity-aware Image Captioning

In this paper we propose VisualNews-Captioner, an entity-aware model for...

The COVID That Wasn't: Counterfactual Journalism Using GPT

In this paper, we explore the use of large language models to assess hum...

The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generation

News articles, image captions, product reviews and many other texts ment...

Where Did the President Visit Last Week? Detecting Celebrity Trips from News Articles

Celebrities' whereabouts are of pervasive importance. For instance, wher...

Understanding Politics via Contextualized Discourse Processing

Politicians often have underlying agendas when reacting to events. Argum...

GateNLP-UShef at SemEval-2022 Task 8: Entity-Enriched Siamese Transformer for Multilingual News Article Similarity

This paper describes the second-placed system on the leaderboard of SemE...

Joint Event Detection and Entity Resolution: a Virtuous Cycle

Clustering web documents has numerous applications, such as aggregating ...

Please sign up or login with your details

Forgot password? Click here to reset