A submodular-supermodular procedure with applications to discriminative structure learning

07/04/2012
by   Mukund Narasimhan, et al.
0

In this paper, we present an algorithm for minimizing the difference between two submodular functions using a variational framework which is based on (an extension of) the concave-convex procedure [17]. Because several commonly used metrics in machine learning, like mutual information and conditional mutual information, are submodular, the problem of minimizing the difference of two submodular problems arises naturally in many machine learning applications. Two such applications are learning discriminatively structured graphical models and feature selection under computational complexity constraints. A commonly used metric for measuring discriminative capacity is the EAR measure which is the difference between two conditional mutual information terms. Feature selection taking complexity considerations into account also fall into this framework because both the information that a set of features provide and the cost of computing and using the features can be modeled as submodular functions. This problem is NP-hard, and we give a polynomial time heuristic for it. We also present results on synthetic data to show that classifiers based on discriminative graphical models using this algorithm can significantly outperform classifiers based on generative graphical models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/09/2014

Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications

We extend the work of Narasimhan and Bilmes [30] for minimizing set func...
research
06/27/2020

Concave Aspects of Submodular Functions

Submodular Functions are a special class of set functions, which general...
research
06/27/2020

Submodular Combinatorial Information Measures with Applications in Machine Learning

Information-theoretic quantities like entropy and mutual information hav...
research
04/30/2019

Categorical Feature Compression via Submodular Optimization

In the era of big data, learning from categorical features with very lar...
research
07/10/2014

A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation

A key problem in statistics and machine learning is the determination of...
research
11/14/2022

The Best Path Algorithm automatic variables selection via High Dimensional Graphical Models

This paper proposes a new algorithm for an automatic variable selection ...
research
01/31/2022

Submodularity In Machine Learning and Artificial Intelligence

In this manuscript, we offer a gentle review of submodularity and superm...

Please sign up or login with your details

Forgot password? Click here to reset