DeepAI AI Chat
Log In Sign Up

Latent Mappings: Generating Open-Ended Expressive Mappings Using Variational Autoencoders

by   Tim Murray-Browne, et al.

In many contexts, creating mappings for gestural interactions can form part of an artistic process. Creators seeking a mapping that is expressive, novel, and affords them a sense of authorship may not know how to program it up in a signal processing patch. Tools like Wekinator and MIMIC allow creators to use supervised machine learning to learn mappings from example input/output pairings. However, a creator may know a good mapping when they encounter it yet start with little sense of what the inputs or outputs should be. We call this an open-ended mapping process. Addressing this need, we introduce the latent mapping, which leverages the latent space of an unsupervised machine learning algorithm such as a Variational Autoencoder trained on a corpus of unlabelled gestural data from the creator. We illustrate it with Sonified Body, a system mapping full-body movement to sound which we explore in a residency with three dancers.


Tools for Analysis of Shannon-Kotel'nikov Mappings

This document summarizes results on S-K mappings obtained since 2017....

Transferability-based Chain Motion Mapping from Humans to Humanoids for Teleoperation

Although data-driven motion mapping methods are promising to allow intui...

Phoneme-to-viseme mappings: the good, the bad, and the ugly

Visemes are the visual equivalent of phonemes. Although not precisely de...

Modal Uncertainty Estimation via Discrete Latent Representation

Many important problems in the real world don't have unique solutions. I...

LatentPoison - Adversarial Attacks On The Latent Space

Robustness and security of machine learning (ML) systems are intertwined...

Perturbation theory approach to study the latent space degeneracy of Variational Autoencoders

The use of Variational Autoencoders in different Machine Learning tasks ...

SPMoE: Generate Multiple Pattern-Aware Outputs with Sparse Pattern Mixture of Experts

Many generation tasks follow a one-to-many mapping relationship: each in...