DeepAI AI Chat
Log In Sign Up

Learning High Order Feature Interactions with Fine Control Kernels

by   Hristo Paskov, et al.
Columbia University
Stanford University

We provide a methodology for learning sparse statistical models that use as features all possible multiplicative interactions among an underlying atomic set of features. While the resulting optimization problems are exponentially sized, our methodology leads to algorithms that can often solve these problems exactly or provide approximate solutions based on combining highly correlated features. We also introduce an algorithmic paradigm, the Fine Control Kernel framework, so named because it is based on Fenchel Duality and is reminiscent of kernel methods. Its theory is tailored to large sparse learning problems, and it leads to efficient feature screening rules for interactions. These rules are inspired by the Apriori algorithm for market basket analysis – which also falls under the purview of Fine Control Kernels, and can be applied to a plurality of learning problems including the Lasso and sparse matrix estimation. Experiments on biomedical datasets demonstrate the efficacy of our methodology in deriving algorithms that efficiently produce interactions models which achieve state-of-the-art accuracy and are interpretable.


page 1

page 2

page 3

page 4


Screening Rules for Convex Problems

We propose a new framework for deriving screening rules for convex optim...

Safe Feature Pruning for Sparse High-Order Interaction Models

Taking into account high-order interactions among covariates is valuable...

Correlated Feature Selection with Extended Exclusive Group Lasso

In many high dimensional classification or regression problems set in a ...

From safe screening rules to working sets for faster Lasso-type solvers

Convex sparsity-promoting regularizations are ubiquitous in modern stati...

Regularization Techniques for Learning with Matrices

There is growing body of learning problems for which it is natural to or...

Improved Random Features for Dot Product Kernels

Dot product kernels, such as polynomial and exponential (softmax) kernel...

Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression

High-order parametric models that include terms for feature interactions...