DeepAI AI Chat
Log In Sign Up

Learning High Order Feature Interactions with Fine Control Kernels

02/09/2020
by   Hristo Paskov, et al.
EPFL
Columbia University
Stanford University
0

We provide a methodology for learning sparse statistical models that use as features all possible multiplicative interactions among an underlying atomic set of features. While the resulting optimization problems are exponentially sized, our methodology leads to algorithms that can often solve these problems exactly or provide approximate solutions based on combining highly correlated features. We also introduce an algorithmic paradigm, the Fine Control Kernel framework, so named because it is based on Fenchel Duality and is reminiscent of kernel methods. Its theory is tailored to large sparse learning problems, and it leads to efficient feature screening rules for interactions. These rules are inspired by the Apriori algorithm for market basket analysis – which also falls under the purview of Fine Control Kernels, and can be applied to a plurality of learning problems including the Lasso and sparse matrix estimation. Experiments on biomedical datasets demonstrate the efficacy of our methodology in deriving algorithms that efficiently produce interactions models which achieve state-of-the-art accuracy and are interpretable.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/23/2016

Screening Rules for Convex Problems

We propose a new framework for deriving screening rules for convex optim...
06/26/2015

Safe Feature Pruning for Sparse High-Order Interaction Models

Taking into account high-order interactions among covariates is valuable...
02/27/2020

Correlated Feature Selection with Extended Exclusive Group Lasso

In many high dimensional classification or regression problems set in a ...
03/21/2017

From safe screening rules to working sets for faster Lasso-type solvers

Convex sparsity-promoting regularizations are ubiquitous in modern stati...
10/04/2009

Regularization Techniques for Learning with Matrices

There is growing body of learning problems for which it is natural to or...
01/21/2022

Improved Random Features for Dot Product Kernels

Dot product kernels, such as polynomial and exponential (softmax) kernel...
12/25/2017

Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression

High-order parametric models that include terms for feature interactions...