Differentiable Projection for Constrained Deep Learning

11/21/2021
by   Dou Huang, et al.
0

Deep neural networks (DNNs) have achieved extraordinary performance in solving different tasks in various fields. However, the conventional DNN model is steadily approaching the ground-truth value through loss backpropagation. In some applications, some prior knowledge could be easily obtained, such as constraints which the ground truth observation follows. Here, we try to give a general approach to incorporate information from these constraints to enhance the performance of the DNNs. Theoretically, we could formulate these kinds of problems as constrained optimization problems that KKT conditions could solve. In this paper, we propose to use a differentiable projection layer in DNN instead of directly solving time-consuming KKT conditions. The proposed projection method is differentiable, and no heavy computation is required. Finally, we also conducted some experiments using a randomly generated synthetic dataset and image segmentation task using the PASCAL VOC dataset to evaluate the performance of the proposed projection method. Experimental results show that the projection method is sufficient and outperforms baseline methods.

READ FULL TEXT
research
05/31/2023

Power Control with QoS Guarantees: A Differentiable Projection-based Unsupervised Learning Framework

Deep neural networks (DNNs) are emerging as a potential solution to solv...
research
06/30/2020

Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey

While Deep Neural Networks (DNNs) achieve state-of-the-art results in ma...
research
04/30/2023

Detecting Novelties with Empty Classes

For open world applications, deep neural networks (DNNs) need to be awar...
research
08/21/2023

Differentiable Frank-Wolfe Optimization Layer

Differentiable optimization has received a significant amount of attenti...
research
01/29/2019

A Parallel Projection Method for Metric Constrained Optimization

Many clustering applications in machine learning and data mining rely on...
research
07/26/2020

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance a...
research
10/19/2015

Qualitative Projection Using Deep Neural Networks

Deep neural networks (DNN) abstract by demodulating the output of linear...

Please sign up or login with your details

Forgot password? Click here to reset