Joint Continuous and Discrete Model Selection via Submodularity

02/17/2021
by   Jonathan Bunton, et al.
0

In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenarios, however, the meaningful structure is specified in some discrete space, leading to difficult nonconvex optimization problems. In this paper, we relate the model selection problem with structure-promoting regularizers to submodular function minimization defined with continuous and discrete arguments. In particular, we leverage submodularity theory to identify a class of these problems that can be solved exactly and efficiently with an agnostic combination of discrete and continuous optimization routines. We show how simple continuous or discrete constraints can also be handled for certain problem classes, motivated by robust optimization. Finally, we numerically validate our theoretical results with several proof-of-concept examples, comparing against state-of-the-art algorithms.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/26/2013

Structured Convex Optimization under Submodular Constraints

A number of discrete and continuous optimization problems in machine lea...
02/26/2019

A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems

We are motivated by large scale submodular optimization problems, where ...
09/16/2019

Improved estimation via model selection method for semimartingale regressions based on discrete data

We consider the robust adaptive nonparametric estimation problem for a p...
02/28/2017

Robust Budget Allocation via Continuous Submodular Functions

The optimal allocation of resources for maximizing influence, spread of ...
07/30/2013

Multi-dimensional Parametric Mincuts for Constrained MAP Inference

In this paper, we propose novel algorithms for inferring the Maximum a P...
03/06/2018

Model Selection as a Multiple Testing Procedure: Improving Akaike's Information Criterion

By interpreting the model selection problem as a multiple hypothesis tes...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.