Eigen-Stratified Models

01/27/2020
by   Jonathan Tuck, et al.
6

Stratified models depend in an arbitrary way on a selected categorical feature that takes K values, and depend linearly on the other n features. Laplacian regularization with respect to a graph on the feature values can greatly improve the performance of a stratified model, especially in the low-data regime. A significant issue with Laplacian-regularized stratified models is that the model is K times the size of the base model, which can be quite large. We address this issue by formulating eigen-stratifed models, which are stratified models with an additional constraint that the model parameters are linear combinations of some modest number m of bottom eigenvectors of the graph Laplacian, i.e., those associated with the m smallest eigenvalues. With eigen-stratified models, we only need to store the m bottom eigenvectors and the corresponding coefficients as the stratified model parameters. This leads to a reduction, sometimes large, of model size when m ≤ n and m ≪ K. In some cases, the additional regularization implicit in eigen-stratified models can improve out-of-sample performance over standard Laplacian regularized stratified models.

READ FULL TEXT

page 12

page 14

page 15

page 16

page 18

research
04/26/2019

A Distributed Method for Fitting Laplacian Regularized Stratified Models

Stratified models are models that depend in an arbitrary way on a set of...
research
05/04/2020

Fitting Laplacian Regularized Stratified Gaussian Models

We consider the problem of jointly estimating multiple related zero-mean...
research
10/20/2019

Spectral bounds of the regularized normalized Laplacian for random geometric graphs

In this work, we study the spectrum of the regularized normalized Laplac...
research
05/04/2023

Joint Graph Learning and Model Fitting in Laplacian Regularized Stratified Models

Laplacian regularized stratified models (LRSM) are models that utilize t...
research
11/09/2022

A Unified Analysis of Multi-task Functional Linear Regression Models with Manifold Constraint and Composite Quadratic Penalty

This work studies the multi-task functional linear regression models whe...
research
09/12/2022

FiBiNet++:Improving FiBiNet by Greatly Reducing Model Size for CTR Prediction

Click-Through Rate(CTR) estimation has become one of the most fundamenta...
research
10/08/2011

Regularized Laplacian Estimation and Fast Eigenvector Approximation

Recently, Mahoney and Orecchia demonstrated that popular diffusion-based...

Please sign up or login with your details

Forgot password? Click here to reset