Utilizing Static Analysis and Code Generation to Accelerate Neural Networks

06/27/2012
by   Lawrence McAfee, et al.
0

As datasets continue to grow, neural network (NN) applications are becoming increasingly limited by both the amount of available computational power and the ease of developing high-performance applications. Researchers often must have expert systems knowledge to make their algorithms run efficiently. Although available computing power increases rapidly each year, algorithm efficiency is not able to keep pace due to the use of general purpose compilers, which are not able to fully optimize specialized application domains. Within the domain of NNs, we have the added knowledge that network architecture remains constant during training, meaning the architecture's data structure can be statically optimized by a compiler. In this paper, we present SONNC, a compiler for NNs that utilizes static analysis to generate optimized parallel code. We show that SONNC's use of static optimizations make it able to outperform hand-optimized C++ code by up to 7.8X, and MATLAB code by up to 24X. Additionally, we show that use of SONNC significantly reduces code complexity when using structurally sparse networks.

READ FULL TEXT
research
07/09/2018

Architecture and performance of Devito, a system for automated stencil computation

Stencil computations are a key part of many high-performance computing a...
research
04/16/2019

swTVM: Exploring the Automated Compilation for Deep Learning on Sunway Architecture

The flourish of deep learning frameworks and hardware platforms has been...
research
03/15/2023

PyExaFMM: an exercise in designing high-performance software with Python and Numba

Numba is a game-changing compiler for high-performance computing with Py...
research
01/21/2021

Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors

Focal-plane Sensor-processors (FPSPs) are a camera technology that enabl...
research
12/01/2022

BaCO: A Fast and Portable Bayesian Compiler Optimization Framework

We introduce the Bayesian Compiler Optimization framework (BaCO), a gene...
research
10/17/2019

A Tool for Automatically Suggesting Source-Code Optimizations for Complex GPU Kernels

Future computing systems, from handhelds to supercomputers, will undoubt...
research
06/13/2019

A JIT Compiler for Neural Network Inference

This paper describes a C++ library that compiles neural network models a...

Please sign up or login with your details

Forgot password? Click here to reset