GPU-Accelerated Forward-Backward algorithm with Application to Lattice-Free MMI

10/22/2021
by   Lucas Ondel, et al.
0

We propose to express the forward-backward algorithm in terms of operations between sparse matrices in a specific semiring. This new perspective naturally leads to a GPU-friendly algorithm which is easy to implement in Julia or any programming languages with native support of semiring algebra. We use this new implementation to train a TDNN with the LF-MMI objective function and we compare the training time of our system with PyChain - a recently introduced C++/CUDA implementation of the LF-MMI loss. Our implementation is about two times faster while not having to use any approximation such as the "leaky-HMM".

READ FULL TEXT
research
05/26/2023

Emergent representations in networks trained with the Forward-Forward algorithm

The Backpropagation algorithm, widely used to train neural networks, has...
research
05/01/2016

Further properties of the forward-backward envelope with applications to difference-of-convex programming

In this paper, we further study the forward-backward envelope first intr...
research
11/08/2016

A new GPU implementation for lattice-Boltzmann simulations on sparse geometries

We describe a high-performance implementation of the lattice Boltzmann m...
research
06/16/2018

A New High Performance and Scalable SVD algorithm on Distributed Memory Systems

This paper introduces a high performance implementation of Zolo-SVD algo...
research
01/30/2015

Montblanc: GPU accelerated Radio Interferometer Measurement Equations in support of Bayesian Inference for Radio Observations

We present Montblanc, a GPU implementation of the Radio interferometer m...
research
04/09/2018

A GPU-based WFST Decoder with Exact Lattice Generation

We describe initial work on an extension of the Kaldi toolkit that suppo...
research
06/11/2021

Time Warps, from Algebra to Algorithms

Graded modalities have been proposed in recent work on programming langu...

Please sign up or login with your details

Forgot password? Click here to reset