LU-Net: Invertible Neural Networks Based on Matrix Factorization

02/21/2023
βˆ™
by   Robin Chan, et al.
βˆ™
0
βˆ™

LU-Net is a simple and fast architecture for invertible neural networks (INN) that is based on the factorization of quadratic weight matrices 𝖠=𝖫𝖴, where 𝖫 is a lower triangular matrix with ones on the diagonal and 𝖴 an upper triangular matrix. Instead of learning a fully occupied matrix 𝖠, we learn 𝖫 and 𝖴 separately. If combined with an invertible activation function, such layers can easily be inverted whenever the diagonal entries of 𝖴 are different from zero. Also, the computation of the determinant of the Jacobian matrix of such layers is cheap. Consequently, the LU architecture allows for cheap computation of the likelihood via the change of variables formula and can be trained according to the maximum likelihood principle. In our numerical experiments, we test the LU-net architecture as generative model on several academic datasets. We also provide a detailed comparison with conventional invertible neural networks in terms of performance, training as well as run time.

READ FULL TEXT

page 5

page 6

page 7

page 10

research
βˆ™ 12/25/2022

FMM-Net: neural network architecture based on the Fast Multipole Method

In this paper, we propose a new neural network architecture based on the...
research
βˆ™ 03/10/2020

Off-diagonal Symmetric Nonnegative Matrix Factorization

Symmetric nonnegative matrix factorization (symNMF) is a variant of nonn...
research
βˆ™ 09/27/2019

In-training Matrix Factorization for Parameter-frugal Neural Machine Translation

In this paper, we propose the use of in-training matrix factorization to...
research
βˆ™ 10/24/2022

Fast and Low-Memory Deep Neural Networks Using Binary Matrix Factorization

Despite the outstanding performance of deep neural networks in different...
research
βˆ™ 10/06/2022

A Step Towards Uncovering The Structure of Multistable Neural Networks

We study the structure of multistable recurrent neural networks. The act...
research
βˆ™ 03/03/2017

Decoupled Block-Wise ILU(k) Preconditioner on GPU

This research investigates the implementation mechanism of block-wise IL...
research
βˆ™ 09/26/2014

Order-invariant prior specification in Bayesian factor analysis

In (exploratory) factor analysis, the loading matrix is identified only ...

Please sign up or login with your details

Forgot password? Click here to reset