AI Chat AI Image Generator AI Video Text to Speech

A Note on Zeroth-Order Optimization on the Simplex

08/02/2022

∙

by Tijana Zrnic, et al.

∙

∙

We construct a zeroth-order gradient estimator for a smooth function defined on the probability simplex. The proposed estimator queries the simplex only. We prove that projected gradient descent and the exponential weights algorithm, when run with this estimator instead of exact gradients, converge at a 𝒪(T^-1/4) rate.

Tijana Zrnic
16 publications
Eric Mazumdar
19 publications

page 1

page 2

page 3

page 4

research

∙ 12/13/2017

Potential-Function Proofs for First-Order Methods

This note discusses proofs for convergence of first-order methods based ...

0 Nikhil Bansal, et al. ∙

research

∙ 08/13/2023

Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training

Binarization of neural networks is a dominant paradigm in neural network...

0 Xiao-Ming Wu, et al. ∙

research

∙ 06/29/2020

Natural Gradient for Combined Loss Using Wavelets

Natural gradients have been widely used in optimization of loss function...

0 Lexing Ying, et al. ∙

research

∙ 12/28/2022

Robustifying Markowitz

Markowitz mean-variance portfolios with sample mean and covariance as in...

0 Wolfgang Karl Härdle, et al. ∙

research

∙ 05/31/2020

Tree-Projected Gradient Descent for Estimating Gradient-Sparse Parameters on Graphs

We study estimation of a gradient-sparse parameter vector θ^* ∈ℝ^p, havi...

0 Sheng Xu, et al. ∙

research

∙ 09/08/2022

Stochastic gradient descent with gradient estimator for categorical features

Categorical data are present in key areas such as health or supply chain...

0 Paul Peseux, et al. ∙

research

∙ 02/02/2022

Do Differentiable Simulators Give Better Policy Gradients?

Differentiable simulators promise faster computation time for reinforcem...

0 H. J. Terry Suh, et al. ∙