DeepAI AI Chat
Log In Sign Up

MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities

by   Jason Armitage, et al.

In this paper, we introduce the MLM (Multiple Languages and Modalities) dataset - a new resource to train and evaluate multitask systems on samples in multiple modalities and three languages. The generation process and inclusion of semantic data provide a resource that further tests the ability for multitask systems to learn relationships between entities. The dataset is designed for researchers and developers who build applications that perform multiple tasks on data encountered on the web and in digital archives. A second version of MLM provides a geo-representative subset of the data with weighted samples for countries of the European Union. We demonstrate the value of the resource in developing novel applications in the digital humanities with a motivating use case and specify a benchmark set of tasks to retrieve modalities and locate entities in the dataset. Evaluation of baseline multitask and single task systems on the full and geo-representative versions of MLM demonstrate the challenges of generalising on diverse data. In addition to the digital humanities, we expect the resource to contribute to research in multimodal representation learning, location estimation, and scene understanding.


HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

Learning multimodal representations involves discovering correspondences...

The Benefit of Multitask Representation Learning

We discuss a general method to learn data representations from multiple ...

Mutlitask Learning for Cross-Lingual Transfer of Semantic Dependencies

We describe a method for developing broad-coverage semantic dependency p...

A multitask deep learning model for real-time deployment in embedded systems

We propose an approach to Multitask Learning (MTL) to make deep learning...

Benchmark Environments for Multitask Learning in Continuous Domains

As demand drives systems to generalize to various domains and problems, ...

MultiTask-CenterNet (MCN): Efficient and Diverse Multitask Learning using an Anchor Free Approach

Multitask learning is a common approach in machine learning, which allow...

Look, Read and Feel: Benchmarking Ads Understanding with Multimodal Multitask Learning

Given the massive market of advertising and the sharply increasing onlin...