Eluder Dimension and Generalized Rank

We study the relationship between the eluder dimension for a function class and a generalized notion of rank, defined for any monotone "activation" σ : ℝ→ℝ, which corresponds to the minimal dimension required to represent the class as a generalized linear model. When σ has derivatives bounded away from 0, it is known that σ-rank gives rise to an upper bound on eluder dimension for any function class; we show however that eluder dimension can be exponentially smaller than σ-rank. We also show that the condition on the derivative is necessary; namely, when σ is the relu activation, we show that eluder dimension can be exponentially larger than σ-rank.

Authors

• 1 publication
• 6 publications
• 27 publications
• 75 publications
• A Linear Upper Bound on the Weisfeiler-Leman Dimension of Graphs of Bounded Genus

The Weisfeiler-Leman (WL) dimension of a graph is a measure for the inhe...
04/15/2019 ∙ by Martin Grohe, et al. ∙ 0

• Valued rank-metric codes

In this paper, we study linear spaces of matrices defined over discretel...
04/07/2021 ∙ by Yassine El Maazouz, et al. ∙ 0

• Learning to Rank With Bregman Divergences and Monotone Retargeting

This paper introduces a novel approach for learning to rank (LETOR) base...
10/16/2012 ∙ by Sreangsu Acharyya, et al. ∙ 0

• A Polymatroid Approach to Generalized Weights of Rank Metric Codes

We consider the notion of a (q,m)-polymatroid, due to Shiromoto, and the...
04/03/2019 ∙ by Sudhir R. Ghorpade, et al. ∙ 0

• Minimum Width for Universal Approximation

The universal approximation property of width-bounded networks has been ...
06/16/2020 ∙ by Sejun Park, et al. ∙ 0

• Dimension Correction for Hierarchical Latent Class Models

Model complexity is an important factor to consider when selecting among...
12/12/2012 ∙ by Tomas Kocka, et al. ∙ 0