Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark

09/04/2021
by   Zhengcong Fei, et al.
0

As a kind of new expression elements, Internet memes are popular and extensively used in online chatting scenarios since they manage to make dialogues vivid, moving, and interesting. However, most current dialogue researches focus on text-only dialogue tasks. In this paper, we propose a new task named as Meme incorporated Open-domain Dialogue (MOD). Compared to previous dialogue tasks, MOD is much more challenging since it requires the model to understand the multimodal elements as well as the emotions behind them. To facilitate the MOD research, we construct a large-scale open-domain multimodal dialogue dataset incorporating abundant Internet memes into utterances. The dataset consists of ∼45K Chinese conversations with ∼606K utterances. Each conversation contains about 13 utterances with about 4 Internet memes on average and each utterance equipped with an Internet meme is annotated with the corresponding emotion. In addition, we present a simple and effective method, which utilizes a unified generation network to solve the MOD task. Experimental results demonstrate that our method trained on the proposed corpus is able to achieve expressive communication including texts and memes. The corpus and models have been publicly available at https://github.com/lizekang/DSTC10-MOD.

READ FULL TEXT
research
03/08/2022

Towards Building an Open-Domain Dialogue System Incorporated with Internet Memes

In recent years, Internet memes have been widely used in online chatting...
research
06/05/2019

Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)

Sarcasm is often expressed through several verbal and non-verbal cues, e...
research
04/01/2017

Multimodal Dialogs (MMD): A large-scale dataset for studying multimodal domain-aware conversations

While multimodal conversation agents are gaining importance in several d...
research
08/11/2021

DeliData: A dataset for deliberation in multi-party problem solving

Dialogue systems research is traditionally focused on dialogues between ...
research
09/16/2022

Selecting Stickers in Open-Domain Dialogue through Multitask Learning

With the increasing popularity of online chatting, stickers are becoming...
research
11/01/2018

I Know the Feeling: Learning to Converse with Empathy

Beyond understanding what is being discussed, human communication requir...
research
11/22/2019

The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service

Human conversations in real scenarios are complicated and building a hum...

Please sign up or login with your details

Forgot password? Click here to reset