End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning

02/04/2018
by   Panagiotis Tzirakis, et al.
0

We introduce End2You -- the Imperial College London toolkit for multimodal profiling by end-to-end deep learning. End2You is an open-source toolkit implemented in Python and is based on Tensorflow. It provides capabilities to train and evaluate models in an end-to-end manner, i.e., using raw input. It supports input from raw audio, visual, physiological or other types of information or combination of those, and the output can be of an arbitrary representation, for either classification or regression tasks. To our knowledge, this is the first toolkit that provides generic end-to-end learning for profiling capabilities in either unimodal or multimodal cases. To test our toolkit, we utilise the RECOLA database as was used in the AVEC 2016 challenge. Experimental results indicate that End2You can provide comparable results to state-of-the-art methods despite no need of expert-alike feature representations, but self-learning these from the data "end to end".

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/11/2020

deepSELF: An Open Source Deep Self End-to-End Learning Framework

We introduce an open-source toolkit, i.e., the deep Self End-to-end Lear...
research
07/25/2021

MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolbox

We introduce the MuSe-Toolbox - a Python-based open-source toolkit for c...
research
05/22/2016

openXBOW - Introducing the Passau Open-Source Crossmodal Bag-of-Words Toolkit

We introduce openXBOW, an open-source toolkit for the generation of bag-...
research
06/28/2023

MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Learning multimodal representations involves integrating information fro...
research
08/05/2023

PhysioKit: Open-source, Low-cost Physiological Computing Toolkit for Single and Multi-user Studies

The proliferation of physiological sensors opens new opportunities to ex...
research
09/03/2023

End-to-End Learning on Multimodal Knowledge Graphs

Knowledge graphs enable data scientists to learn end-to-end on heterogen...
research
05/13/2022

The ACM Multimedia 2022 Computational Paralinguistics Challenge: Vocalisations, Stuttering, Activity, Mosquitoes

The ACM Multimedia 2022 Computational Paralinguistics Challenge addresse...

Please sign up or login with your details

Forgot password? Click here to reset