Log In Sign Up

Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations

by   Michael Hopkins, et al.

Although double-precision floating-point arithmetic currently dominates high-performance computing, there is increasing interest in smaller and simpler arithmetic types. The main reasons are potential improvements in energy efficiency and memory footprint and bandwidth. However, simply switching to lower-precision types typically results in increased numerical errors. We investigate approaches to improving the accuracy of reduced-precision fixed-point arithmetic types, using examples in an important domain for numerical computation in neuroscience: the solution of Ordinary Differential Equations (ODEs). The Izhikevich neuron model is used to demonstrate that rounding has an important role in producing accurate spike timings from explicit ODE solution algorithms. In particular, fixed-point arithmetic with stochastic rounding consistently results in smaller errors compared to single precision floating-point and fixed-point arithmetic with round-to-nearest across a range of neuron behaviours and ODE solvers. A computationally much cheaper alternative is also investigated, inspired by the concept of dither that is a widely understood mechanism for providing resolution below the least significant bit (LSB) in digital signal processing. These results will have implications for the solution of ODEs in other subject areas, and should also be directly relevant to the huge range of practical problems that are represented by Partial Differential Equations (PDEs).


Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ODEs

Although double-precision floating-point arithmetic currently dominates ...

Fixed-point iterative linear inverse solver with extended precision

Solving linear systems is a ubiquitous task in science and engineering. ...

Deep Neural Networks to Correct Sub-Precision Errors in CFD

Loss of information in numerical simulations can arise from various sour...

Issues with rounding in the GCC implementation of the ISO 18037:2008 standard fixed-point arithmetic

We describe various issues caused by the lack of round-to-nearest mode i...

Stochastic Rounding: Algorithms and Hardware Accelerator

Algorithms and a hardware accelerator for performing stochastic rounding...

Taylor Polynomials in High Arithmetic Precision as Universal Approximators

Function approximation is a generic process in a variety of computationa...

Synthesizing Power and Area Efficient Image Processing Pipelines on FPGAs using Customized Bit-widths

High-level synthesis (HLS) has received significant attention in recent ...