Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

02/06/2015
by   Kaiming He, et al.
Microsoft
0

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94 classification dataset. This is a 26 winner (GoogLeNet, 6.66 human-level performance (5.1 challenge.

READ FULL TEXT
06/01/2016

Improving Deep Neural Network with Multiple Parametric Exponential Linear Units

Activation function is crucial to the recent successes of deep neural ne...
10/17/2021

Network Augmentation for Tiny Deep Learning

We introduce Network Augmentation (NetAug), a new training method for im...
04/21/2018

Study of Residual Networks for Image Recognition

Deep neural networks demonstrate to have a high performance on image cla...
11/17/2017

xUnit: Learning a Spatial Activation Function for Efficient Image Restoration

In recent years, deep neural networks (DNNs) achieved unprecedented perf...
10/19/2018

Leveraging Product as an Activation Function in Deep Networks

Product unit neural networks (PUNNs) are powerful representational model...
02/12/2019

Improving learnability of neural networks: adding supplementary axes to disentangle data representation

Over-parameterized deep neural networks have proven to be able to learn ...
01/20/2022

What can we learn from misclassified ImageNet images?

Understanding the patterns of misclassified ImageNet images is particula...

Code Repositories

ndsb

Kaggle NDSB competition with deep learning


view repo

prelu_net

Implementation of PReLUNet by chainer (Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification: https://arxiv.org/abs/1502.01852)


view repo

CNN_MATLAB

Convolution neural network


view repo

Autonomous_Car

A Raspberry Pi controlled autonomous RC car


view repo

Please sign up or login with your details

Forgot password? Click here to reset