Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery

02/07/2022
by   Nathaniel Trask, et al.
0

We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent space. This approach enables discovery of fingerprints which may be detected in high-dimensional heterogeneous datasets, avoiding traditional bottlenecks related to high-fidelity measurement and characterization. Motivated by accelerated co-design and optimization of materials manufacturing processes, a dataset of lattice metamaterials from metal additive manufacturing demonstrates accurate cross modal inference between images of mesoscale topology and mechanical stress-strain response.

READ FULL TEXT

page 3

page 6

page 7

page 8

research
09/18/2019

Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals

We propose a novel deep training algorithm for joint representation of a...
research
09/14/2019

Joint Wasserstein Autoencoders for Aligning Multimodal Embeddings

One of the key challenges in learning joint embeddings of multiple modal...
research
03/23/2021

Shared Latent Space of Font Shapes and Impressions

We have specific impressions from the style of a typeface (font), sugges...
research
03/30/2018

Cross-modal Deep Variational Hand Pose Estimation

The human hand moves in complex and high-dimensional ways, making estima...
research
09/25/2020

A physics-informed operator regression framework for extracting data-driven continuum models

The application of deep learning toward discovery of data-driven models ...
research
10/28/2022

Multimodal Transformer for Parallel Concatenated Variational Autoencoders

In this paper, we propose a multimodal transformer using parallel concat...
research
03/27/2023

Cross-study analyses of microbial abundance using generalized common factor methods

By creating networks of biochemical pathways, communities of micro-organ...

Please sign up or login with your details

Forgot password? Click here to reset