An Integer Programming Approach to Deep Neural Networks with Binary Activation Functions

07/07/2020
by   Bubacarr Bah, et al.
0

We study deep neural networks with binary activation functions (BDNN), i.e. the activation function only has two states. We show that the BDNN can be reformulated as a mixed-integer linear program which can be solved to global optimality by classical integer programming solvers. Additionally, a heuristic solution algorithm is presented and we study the model under data uncertainty, applying a two-stage robust optimization approach. We implemented our methods on random and real datasets and show that the heuristic version of the BDNN outperforms classical deep neural networks on the Breast Cancer Wisconsin dataset while performing worse on random data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2021

Efficient and Robust Mixed-Integer Optimization Methods for Training Binarized Deep Neural Networks

Compared to classical deep neural networks its binarized versions can be...
research
01/13/2021

Reproducing Activation Function for Deep Learning

In this paper, we propose the reproducing activation function to improve...
research
04/19/2023

Parallel Neural Networks in Golang

This paper describes the design and implementation of parallel neural ne...
research
06/02/2023

Uniform Convergence of Deep Neural Networks with Lipschitz Continuous Activation Functions and Variable Widths

We consider deep neural networks with a Lipschitz continuous activation ...
research
04/29/2023

When Deep Learning Meets Polyhedral Theory: A Survey

In the past decade, deep learning became the prevalent methodology for p...
research
01/23/2023

Topological Understanding of Neural Networks, a survey

We look at the internal structure of neural networks which is usually tr...
research
12/13/2021

Acceleration techniques for optimization over trained neural network ensembles

We study optimization problems where the objective function is modeled t...

Please sign up or login with your details

Forgot password? Click here to reset