Parseval Proximal Neural Networks

by   Marzieh Hasannasab, et al.

The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable Parseval (frame) proximal neural networks (PPNNs). Let H and K be real Hilbert spaces, b ∈K and T ∈B (H,K) a linear operator with closed range and Moore-Penrose inverse T^†. Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator ProxK→K the operator T^† Prox ( T · + b) is a proximity operator on H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator Prox = S_λℓ_2 →ℓ_2 and any frame analysis operator TH→ℓ_2, that the frame shrinkage operator T^† S_λ T is a proximity operator in a suitable Hilbert space. Further, the concatenation of proximity operators on R^d equipped with different norms establishes a PNN. If the network arises from Parseval frame analysis or synthesis operators, it forms an averaged operator, called PPNN. The involved linear operators, respectively their transposed operators, are in a Stiefel manifold, so that minimization methods on Stiefel manifolds can be applied for training such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.



There are no comments yet.


page 1

page 2

page 3

page 4


Strong-consistent autoregressive predictors in abstract Banach spaces

This work derives new results on the strong-consistency of a componentwi...

Strongly consistent autoregressive predictors in abstract Banach spaces

This work derives new results on strong consistent estimation and predic...

Interaction of Multiple Tensor Product Operators of the Same Type: an Introduction

Tensor product operators on finite dimensional Hilbert spaces are studie...

Cospanning characterizations of violator and co-violator spaces

Given a finite set E and an operator sigma:2^E–>2^E, two subsets X,Y of ...

Error Analysis of Douglas-Rachford Algorithm for Linear Inverse Problems: Asymptotics of Proximity Operator for Squared Loss

Proximal splitting-based convex optimization is a promising approach to ...

An optimal linear filter for estimation of random functions in Hilbert space

Let be a square-integrable, zero-mean, random vector with observable re...

The fastest ℓ_1,∞ prox in the west

Proximal operators are of particular interest in optimization problems d...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.