DeepAI AI Chat
Log In Sign Up

CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator

by   Alek Dimitriev, et al.
The University of Texas at Austin

Accurately backpropagating the gradient through categorical variables is a challenging task that arises in various domains, such as training discrete latent variable models. To this end, we propose CARMS, an unbiased estimator for categorical random variables based on multiple mutually negatively correlated (jointly antithetic) samples. CARMS combines REINFORCE with copula based sampling to avoid duplicate samples and reduce its variance, while keeping the estimator unbiased using importance sampling. It generalizes both the ARMS antithetic estimator for binary variables, which is CARMS for two categories, as well as LOORF/VarGrad, the leave-one-out REINFORCE estimator, which is CARMS with independent samples. We evaluate CARMS on several benchmark datasets on a generative modeling task, as well as a structured output prediction task, and find it to outperform competing methods including a strong self-control baseline. The code is publicly available.


page 1

page 2

page 3

page 4


ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variables

Estimating the gradients for binary variables is a task that arises freq...

MuProp: Unbiased Backpropagation for Stochastic Neural Networks

Deep neural networks are powerful parametric models that can be trained ...

ARSM Gradient Estimator for Supervised Learning to Rank

We propose a new model for supervised learning to rank. In our model, th...

Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories

For machine learning models trained with limited labeled training data, ...

Categorical Reparameterization with Gumbel-Softmax

Categorical variables are a natural choice for representing discrete str...

ARSM: Augment-REINFORCE-Swap-Merge Estimator for Gradient Backpropagation Through Categorical Variables

To address the challenge of backpropagating the gradient through categor...

Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators

Discrete and especially binary random variables occur in many machine le...