Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication

11/11/2019
by   Ruize Wang, et al.
20

Visual storytelling aims to generate a narrative paragraph from a sequence of images automatically. Existing approaches construct text description independently for each image and roughly concatenate them as a story, which leads to the problem of generating semantically incoherent content. In this paper, we proposed a new way for visual storytelling by introducing a topic description task to detect the global semantic context of an image stream. A story is then constructed with the guidance of the topic description. In order to combine the two generation tasks, we propose a multi-agent communication framework that regards the topic description generator and the story generator as two agents and learn them simultaneously via iterative updating mechanism. We validate our approach on VIST, where quantitative results, ablations, and human evaluation demonstrate our method's good ability in generating stories with higher quality compared to state-of-the-art methods.

READ FULL TEXT

page 1

page 7

page 11

page 12

research
01/11/2019

From Plots to Endings: A Reinforced Pointer Generator for Story Ending Generation

We introduce a new task named Story Ending Generation (SEG), whic-h aims...
research
09/13/2021

Explain Me the Painting: Multi-Topic Knowledgeable Art Description Generation

Have you ever looked at a painting and wondered what is the story behind...
research
08/11/2020

Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling

Visual Storytelling (VIST) is a task to tell a narrative story about a c...
research
12/14/2021

TopNet: Learning from Neural Topic Model to Generate Long Stories

Long story generation (LSG) is one of the coveted goals in natural langu...
research
08/17/2023

Text-Only Training for Visual Storytelling

Visual storytelling aims to generate a narrative based on a sequence of ...
research
03/10/2022

Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling

As a technically challenging topic, visual storytelling aims at generati...
research
06/03/2018

Contextualize, Show and Tell: A Neural Visual Storyteller

We present a neural model for generating short stories from image sequen...

Please sign up or login with your details

Forgot password? Click here to reset