Parseval Proximal Neural Networks

12/19/2019
by   Marzieh Hasannasab, et al.
0

The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable Parseval (frame) proximal neural networks (PPNNs). Let H and K be real Hilbert spaces, b ∈K and T ∈B (H,K) a linear operator with closed range and Moore-Penrose inverse T^†. Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator ProxK→K the operator T^† Prox ( T · + b) is a proximity operator on H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator Prox = S_λℓ_2 →ℓ_2 and any frame analysis operator TH→ℓ_2, that the frame shrinkage operator T^† S_λ T is a proximity operator in a suitable Hilbert space. Further, the concatenation of proximity operators on R^d equipped with different norms establishes a PNN. If the network arises from Parseval frame analysis or synthesis operators, it forms an averaged operator, called PPNN. The involved linear operators, respectively their transposed operators, are in a Stiefel manifold, so that minimization methods on Stiefel manifolds can be applied for training such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

01/26/2018

Strong-consistent autoregressive predictors in abstract Banach spaces

This work derives new results on the strong-consistency of a componentwi...
08/05/2018

Strongly consistent autoregressive predictors in abstract Banach spaces

This work derives new results on strong consistent estimation and predic...
06/29/2021

Interaction of Multiple Tensor Product Operators of the Same Type: an Introduction

Tensor product operators on finite dimensional Hilbert spaces are studie...
05/11/2021

Cospanning characterizations of violator and co-violator spaces

Given a finite set E and an operator sigma:2^E–>2^E, two subsets X,Y of ...
03/18/2021

Error Analysis of Douglas-Rachford Algorithm for Linear Inverse Problems: Asymptotics of Proximity Operator for Squared Loss

Proximal splitting-based convex optimization is a promising approach to ...
08/28/2020

An optimal linear filter for estimation of random functions in Hilbert space

Let be a square-integrable, zero-mean, random vector with observable re...
10/09/2019

The fastest ℓ_1,∞ prox in the west

Proximal operators are of particular interest in optimization problems d...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.