DeepAI AI Chat
Log In Sign Up

Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach

by   Arpan Biswas, et al.

Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g., balancing the Kullback Leibler (KL) and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive to train models, here we explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrate for joint-VAE with rotational invariances. We demonstrate an application of this method for finding joint discrete and continuous rotationally invariant representations for MNIST and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter tuning or trajectory optimization of other ML models.


page 18

page 19

page 20

page 21

page 23

page 24

page 31

page 32


Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax

Deep learning models are full of hyperparameters, which are set manually...

Fast Hyperparameter Tuning using Bayesian Optimization with Directional Derivatives

In this paper we develop a Bayesian optimization based hyperparameter tu...

TzK Flow - Conditional Generative Model

We introduce TzK (pronounced "task"), a conditional flow-based encoder/d...

Bayesian Optimization in Variational Latent Spaces with Dynamic Compression

Data-efficiency is crucial for autonomous robots to adapt to new tasks a...

Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve

Variational autoencoders (VAEs) are powerful tools for learning latent r...

FLO: Fast and Lightweight Hyperparameter Optimization for AutoML

Integrating ML models in software is of growing interest. Building accur...

Optimizing a Digital Twin for Fault Diagnosis in Grid Connected Inverters – A Bayesian Approach

In this paper, a hyperparameter tuning based Bayesian optimization of di...