The Expected Jacobian Outerproduct: Theory and Empirics

by   Shubhendu Trivedi, et al.

The expected gradient outerproduct (EGOP) of an unknown regression function is an operator that arises in the theory of multi-index regression, and is known to recover those directions that are most relevant to predicting the output. However, work on the EGOP, including that on its cheap estimators, is restricted to the regression setting. In this work, we adapt this operator to the multi-class setting, which we dub the expected Jacobian outerproduct (EJOP). Moreover, we propose a simple rough estimator of the EJOP and show that somewhat surprisingly, it remains statistically consistent under mild assumptions. Furthermore, we show that the eigenvalues and eigenspaces also remain consistent. Finally, we show that the estimated EJOP can be used as a metric to yield improvements in real-world non-parametric classification tasks: both by its use as a metric, and also as cheap initialization in metric learning tasks.



There are no comments yet.


page 13


Consistent Classification Algorithms for Multi-class Non-Decomposable Performance Metrics

We study consistency of learning algorithms for a multi-class performanc...

Kernel Regression with Sparse Metric Learning

Kernel regression is a popular non-parametric fitting technique. It aims...

A Sieve Stochastic Gradient Descent Estimator for Online Nonparametric Regression in Sobolev ellipsoids

The goal of regression is to recover an unknown underlying function that...

Yield curve and macroeconomy interaction: evidence from the non-parametric functional lagged regression approach

Viewing a yield curve as a sparse collection of measurements on a latent...

Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions

Generalised linear models for multi-class classification problems are on...

Tutorial: Deriving The Efficient Influence Curve for Large Models

This paper aims to provide a tutorial for upper level undergraduate and ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.