Efficient Neural Network Implementation with Quadratic Neuron

11/21/2020

∙

Previous works proved that the combination of the linear neuron network with nonlinear activation functions (e.g. ReLu) can achieve nonlinear function approximation. However, simply widening or deepening the network structure will introduce some training problems. In this work, we are aiming to build a comprehensive second-order CNN implementation framework that includes neuron/network design and system deployment optimization.

READ FULL TEXT

Efficient Neural Network Implementation with Quadratic Neuron

Differential Equation Units: Learning Functional Forms of Activation Functions from Data

Central and Non-central Limit Theorems arising from the Scattering Transform and its Neural Activation Generalization

QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

NetPyNE implementation and rescaling of the Potjans-Diesmanncortical microcircuit model

Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification

A Note on Topology Preservation in Classification, and the Construction of a Universal Neuron Grid

Learning a Single Neuron with Gradient Methods

Efficient Neural Network Implementation with Quadratic Neuron

Related Research

Differential Equation Units: Learning Functional Forms of Activation Functions from Data

Central and Non-central Limit Theorems arising from the Scattering Transform and its Neural Activation Generalization

QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration

NetPyNE implementation and rescaling of the Potjans-Diesmanncortical microcircuit model

Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification

A Note on Topology Preservation in Classification, and the Construction of a Universal Neuron Grid

Learning a Single Neuron with Gradient Methods