Improving learnability of neural networks: adding supplementary axes to disentangle data representation

02/12/2019
by   Kim Bukweon, et al.
4

Over-parameterized deep neural networks have proven to be able to learn an arbitrary dataset with 100% training accuracy. Because of a risk of overfitting and computational cost issues, we cannot afford to increase the number of network nodes if we want achieve better training results for medical images. Previous deep learning research shows that the training ability of a neural network improves dramatically (for the same epoch of training) when a few nodes with supplementary information are added to the network. These few informative nodes allow the network to learn features that are otherwise difficult to learn by generating a disentangled data representation. This paper analyzes how concatenation of additional information as supplementary axes affects the training of the neural networks. This analysis was conducted for a simple multilayer perceptron (MLP) classification model with a rectified linear unit (ReLU) on two-dimensional training data. We compared the networks with and without concatenation of supplementary information to support our analysis. The model with concatenation showed more robust and accurate training results compared to the model without concatenation. We also confirmed that our findings are valid for deeper convolutional neural networks (CNN) using ultrasound images and for a conditional generative adversarial network (cGAN) using the MNIST data.

READ FULL TEXT

page 1

page 4

page 6

research
04/21/2018

Study of Residual Networks for Image Recognition

Deep neural networks demonstrate to have a high performance on image cla...
research
04/14/2018

OmicsMapNet: Transforming omics data to take advantage of Deep Convolutional Neural Network for discovery

We developed OmicsMapNet approach to take advantage of existing deep lea...
research
03/31/2021

Spectral decoupling allows training transferable neural networks in medical imaging

Deep neural networks show impressive performance in medical imaging task...
research
02/28/2019

Poster: On the Feasibility of Training Neural Networks with Visibly Watermarked Dataset

As there are increasing needs of sharing data for machine learning, ther...
research
07/25/2018

Conditional Information Gain Networks

Deep neural network models owe their representational power to the high ...
research
01/09/2019

A Constructive Approach for One-Shot Training of Neural Networks Using Hypercube-Based Topological Coverings

In this paper we presented a novel constructive approach for training de...
research
02/06/2015

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Rectified activation units (rectifiers) are essential for state-of-the-a...

Please sign up or login with your details

Forgot password? Click here to reset