DeepAI AI Chat
Log In Sign Up

LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts

by   Shuming Ma, et al.
Peking University

We introduce the task of automatic live commenting. Live commenting, which is also called `video barrage', is an emerging feature on online video sites that allows real-time comments from viewers to fly across the screen like bullets or roll at the right side of the screen. The live comments are a mixture of opinions for the video and the chit chats with other comments. Automatic live commenting requires AI agents to comprehend the videos and interact with human viewers who also make the comments, so it is a good testbed of an AI agent's ability of dealing with both dynamic vision and language. In this work, we construct a large-scale live comment dataset with 2,361 videos and 895,929 live comments. Then, we introduce two neural models to generate live comments based on the visual and textual contexts, which achieve better performance than previous neural baselines such as the sequence-to-sequence model. Finally, we provide a retrieval-based evaluation protocol for automatic live commenting where the model is asked to sort a set of candidate comments based on the log-likelihood score, and evaluated on metrics such as mean-reciprocal-rank. Putting it all together, we demonstrate the first `LiveBot'.


page 2

page 3


Live Video Comment Generation Based on Surrounding Frames and Live Comments

In this paper, we propose the task of live comment generation. Live comm...

Response to LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts

Live video commenting systems are an emerging feature of online video si...

Comparing Video Based Shoulder Surfing with Live Simulation

We analyze the claims that video recreations of shoulder surfing attacks...

Diversified Co-Attention towards Informative Live Video Commenting

We focus on the task of Automatic Live Video Commenting (ALVC), which ai...

Interactive Variance Attention based Online Spoiler Detection for Time-Sync Comments

Nowadays, time-sync comment (TSC), a new form of interactive comments, h...

Generating Persuasive Visual Storylines for Promotional Videos

Video contents have become a critical tool for promoting products in E-c...

CS-lol: a Dataset of Viewer Comment with Scene in E-sports Live-streaming

Billions of live-streaming viewers share their opinions on scenes they a...