Partition-based formulations for mixed-integer optimization of trained ReLU neural networks

02/08/2021
by   Calvin Tsay, et al.
0

This paper introduces a class of mixed-integer formulations for trained ReLU neural networks. The approach balances model size and tightness by partitioning node inputs into a number of groups and forming the convex hull over the partitions via disjunctive programming. At one extreme, one partition per input recovers the convex hull of a node, i.e., the tightest possible formulation for each node. For fewer partitions, we develop smaller relaxations that approximate the convex hull, and show that they outperform existing formulations. Specifically, we propose strategies for partitioning variables based on theoretical motivations and validate these strategies using extensive computational experiments. Furthermore, the proposed scheme complements known algorithmic approaches, e.g., optimization-based bound tightening captures dependencies within a partition.

READ FULL TEXT
research
07/29/2018

Convex Hull Formulations for Mixed-Integer Multilinear Functions

In this paper, we present convex hull formulations for a mixed-integer, ...
research
02/10/2022

P-split formulations: A class of intermediate formulations between big-M and convex hull for disjunctive constraints

We develop a class of mixed-integer formulations for disjunctive constra...
research
01/03/2022

A Mixed Integer Programming Approach to Training Dense Neural Networks

Artificial Neural Networks (ANNs) are prevalent machine learning models ...
research
05/27/2022

Optimizing Objective Functions from Trained ReLU Neural Networks via Sampling

This paper introduces scalable, sampling-based algorithms that optimize ...
research
01/22/2021

Partition-Based Convex Relaxations for Certifying the Robustness of ReLU Neural Networks

In this paper, we study certifying the robustness of ReLU neural network...
research
01/29/2021

Between steps: Intermediate relaxations between big-M and convex hull formulations

This work develops a class of relaxations in between the big-M and conve...
research
02/20/2023

Model-based feature selection for neural networks: A mixed-integer programming approach

In this work, we develop a novel input feature selection framework for R...

Please sign up or login with your details

Forgot password? Click here to reset