Situation and Behavior Understanding by Trope Detection on Films

by   Chen-Hsi Chang, et al.

The human ability of deep cognitive skills are crucial for the development of various real-world applications that process diverse and abundant user generated input. While recent progress of deep learning and natural language processing have enabled learning system to reach human performance on some benchmarks requiring shallow semantics, such human ability still remains challenging for even modern contextual embedding models, as pointed out by many recent studies. Existing machine comprehension datasets assume sentence-level input, lack of casual or motivational inferences, or could be answered with question-answer bias. Here, we present a challenging novel task, trope detection on films, in an effort to create a situation and behavior understanding for machines. Tropes are storytelling devices that are frequently used as ingredients in recipes for creative works. Comparing to existing movie tag prediction tasks, tropes are more sophisticated as they can vary widely, from a moral concept to a series of circumstances, and embedded with motivations and cause-and-effects. We introduce a new dataset, Tropes in Movie Synopses (TiMoS), with 5623 movie synopses and 95 different tropes collecting from a Wikipedia-style database, TVTropes. We present a multi-stream comprehension network (MulCom) leveraging multi-level attention of words, sentences, and role relations. Experimental result demonstrates that modern models including BERT contextual embedding, movie tag prediction systems, and relational networks, perform at most 37 terms of F1 score. Our MulCom outperforms all modern baselines, by 1.5 to 5.0 F1 score and 1.5 to 3.0 mean of average precision (mAP) score. We also provide a detailed analysis and human evaluation to pave ways for future research.


page 4

page 7

page 10


TrUMAn: Trope Understanding in Movies and Animations

Understanding and comprehending video content is crucial for many real-w...

Evaluating Persian Tokenizers

Tokenization plays a significant role in the process of lexical analysis...

Polish Natural Language Inference and Factivity – an Expert-based Dataset and Benchmarks

Despite recent breakthroughs in Machine Learning for Natural Language Pr...

DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension

We propose DuoRC, a novel dataset for Reading Comprehension (RC) that mo...

Short Text Conversation Based on Deep Neural Network and Analysis on Evaluation Measures

With the development of Natural Language Processing, Automatic question-...

DeepSafety:Multi-level Audio-Text Feature Extraction and Fusion Approach for Violence Detection in Conversations

Natural Language Processing has recently made understanding human intera...

AidUI: Toward Automated Recognition of Dark Patterns in User Interfaces

Past studies have illustrated the prevalence of UI dark patterns, or use...

Please sign up or login with your details

Forgot password? Click here to reset