Improved Convergence Rates for the Orthogonal Greedy Algorithm

06/28/2021 ∙ by Jonathan W. Siegel, et al. ∙ 0

We analyze the orthogonal greedy algorithm when applied to dictionaries 𝔻 whose convex hull has small entropy. We show that if the metric entropy of the convex hull of 𝔻 decays at a rate of O(n^-1/2-α) for α > 0, then the orthogonal greedy algorithm converges at the same rate. This improves upon the well-known O(n^-1/2) convergence rate of the orthogonal greedy algorithm in many cases, most notably for dictionaries corresponding to shallow neural networks. Finally, we show that these improved rates are sharp under the given entropy decay assumptions.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.